[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:24 dist-test-slave-p9q6 krb5kdc[9381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:24 dist-test-slave-p9q6 krb5kdc[9381](info): set up 2 sockets
Sep 27 10:48:24 dist-test-slave-p9q6 krb5kdc[9381](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 10:48:25.959344 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:26.082056 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43097
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:43097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:26.186904 9394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:26.187135 9394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:26.187172 9394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:26.190972 9394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:26.191016 9394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:26.191033 9394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:26.191053 9394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:26.191067 9394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:26.195928 9394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:43097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:26.196805 9394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:26.197671 9394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:26.206290 9400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:26.206279 9394 server_base.cc:1034] running on GCE node
W20260927 10:48:26.206259 9402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.206331 9399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:26.207147 9394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:26.208205 9394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:26.209357 9394 hybrid_clock.cc:648] HybridClock initialized: now 1790506106209326 us; error 44 us; skew 500 ppm
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:26.211961 9394 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:26.212826 9394 webserver.cc:460] Webserver started at http://127.9.39.190:41619/ using document root <none> and password file <none>
I20260927 10:48:26.213349 9394 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:26.213435 9394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:26.213656 9394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:26.215763 9394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "47504058343446c4b9b0295022c7f2cf"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "90e77a5331f021a941935d318a810cf3"
server_key_iv: "f74b3aaa04d8bc18b702b8499ca40d99"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.216354 9394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "47504058343446c4b9b0295022c7f2cf"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "90e77a5331f021a941935d318a810cf3"
server_key_iv: "f74b3aaa04d8bc18b702b8499ca40d99"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.219857 9394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:26.222115 9409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.223294 9394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:26.223431 9394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "47504058343446c4b9b0295022c7f2cf"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "90e77a5331f021a941935d318a810cf3"
server_key_iv: "f74b3aaa04d8bc18b702b8499ca40d99"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.223538 9394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:26.255993 9394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:26.259135 9394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:26.259337 9394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:26.266305 9394 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:43097
I20260927 10:48:26.266325 9461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:43097 every 8 connection(s)
I20260927 10:48:26.267259 9394 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:26.268651 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 9394
I20260927 10:48:26.268812 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:26.270350 9374 external_mini_cluster.cc:1428] Setting key bacd50791bda0b836bb9771ba0ab26d9
I20260927 10:48:26.270526 9462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:26.276330 9462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf: Bootstrap starting.
I20260927 10:48:26.278618 9462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:26.279469 9462 log.cc:824] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:26.281275 9462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf: No bootstrap required, opened a new log
I20260927 10:48:26.285122 9462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47504058343446c4b9b0295022c7f2cf" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43097 } }
I20260927 10:48:26.285313 9462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:26.285346 9462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47504058343446c4b9b0295022c7f2cf, State: Initialized, Role: FOLLOWER
I20260927 10:48:26.285822 9462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47504058343446c4b9b0295022c7f2cf" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43097 } }
I20260927 10:48:26.285961 9462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:26.286027 9462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:26.286152 9462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:26.287031 9462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47504058343446c4b9b0295022c7f2cf" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43097 } }
I20260927 10:48:26.287366 9462 leader_election.cc:302] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47504058343446c4b9b0295022c7f2cf; no voters:
I20260927 10:48:26.287671 9462 leader_election.cc:288] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:26.287807 9467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 1 FOLLOWER]: Leader election won for term 1
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:26.288061 9467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [term 1 LEADER]: Becoming Leader. State: Replica: 47504058343446c4b9b0295022c7f2cf, State: Running, Role: LEADER
I20260927 10:48:26.288394 9467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47504058343446c4b9b0295022c7f2cf" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43097 } }
I20260927 10:48:26.288960 9462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:26.290271 9469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47504058343446c4b9b0295022c7f2cf. Latest consensus state: current_term: 1 leader_uuid: "47504058343446c4b9b0295022c7f2cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47504058343446c4b9b0295022c7f2cf" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43097 } } }
I20260927 10:48:26.290294 9468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47504058343446c4b9b0295022c7f2cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47504058343446c4b9b0295022c7f2cf" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43097 } } }
I20260927 10:48:26.290417 9468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:26.290417 9469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:26.291353 9478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:26.294560 9478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:26.295399 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.273804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42418 (local address 127.9.39.190:43097)
0927 10:48:26.274239 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:26.274258 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:26.275195 (+ 937us) server_negotiation.cc:407] Connection header received
0927 10:48:26.278244 (+ 3049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:26.278263 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:26.278621 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:26.278971 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:26.281225 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.282655 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.283831 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.284525 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.284677 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.289065 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:26.289071 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:26.289085 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:26.289136 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:26.292247 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.292723 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.292728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.292733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.292843 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.293168 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.293172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.293174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.293547 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:26.293753 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:26.294294 (+ 541us) server_negotiation.cc:299] Negotiation successful
0927 10:48:26.294564 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":145,"threads_started":1}
I20260927 10:48:26.301222 9478 catalog_manager.cc:1347] Generated new cluster ID: 3626aeb08d634cdb8d2259efd96ccdb8
I20260927 10:48:26.301285 9478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:26.317586 9478 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:26.318869 9478 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:26.323324 9478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47504058343446c4b9b0295022c7f2cf: Generated new TSK 0
I20260927 10:48:26.324041 9478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:26.354337 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43097
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:26.460966 9488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:26.461197 9488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:26.461237 9488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:26.465091 9488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:26.465142 9488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:26.465197 9488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:26.470270 9488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43097
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:26.471191 9488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:26.472112 9488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:26.481186 9493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.481205 9496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.481251 9494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:26.481566 9488 server_base.cc:1034] running on GCE node
I20260927 10:48:26.482000 9488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:26.482648 9488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:26.483882 9488 hybrid_clock.cc:648] HybridClock initialized: now 1790506106483847 us; error 45 us; skew 500 ppm
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:26.486657 9488 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:26.487587 9488 webserver.cc:460] Webserver started at http://127.9.39.129:33999/ using document root <none> and password file <none>
I20260927 10:48:26.488143 9488 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:26.488237 9488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:26.488480 9488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:26.490681 9488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "39432be13e8d470a92fed13783ac9e1f"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "7e00c462b575afde88f27991c0d4f597"
server_key_iv: "90f4bc18a685c3151d383c18b826e2bf"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.491307 9488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "39432be13e8d470a92fed13783ac9e1f"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "7e00c462b575afde88f27991c0d4f597"
server_key_iv: "90f4bc18a685c3151d383c18b826e2bf"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.495009 9488 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 10:48:26.497406 9503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.498656 9488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:26.498828 9488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "39432be13e8d470a92fed13783ac9e1f"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "7e00c462b575afde88f27991c0d4f597"
server_key_iv: "90f4bc18a685c3151d383c18b826e2bf"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.498942 9488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:26.510926 9488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:26.513633 9488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:26.513839 9488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:26.514391 9488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:26.515278 9488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:26.515353 9488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.515421 9488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:26.515486 9488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.525031 9488 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:45519
I20260927 10:48:26.525051 9616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:45519 every 8 connection(s)
I20260927 10:48:26.525997 9488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:26.529938 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 9488
I20260927 10:48:26.530085 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:26.530351 9374 external_mini_cluster.cc:1428] Setting key 542aee489f5f85f4a2d853bbeafedfbd
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:48:26.540025 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.527978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:37477 (local address 127.9.39.190:43097)
0927 10:48:26.528125 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:26.528130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:26.528919 (+ 789us) server_negotiation.cc:407] Connection header received
0927 10:48:26.530193 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:26.530196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:26.530250 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:26.530373 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:26.531704 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.532213 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.533631 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.533864 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.533950 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.536407 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:26.536413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:26.536419 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:26.536453 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:26.537937 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.538478 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.538521 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.538522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.538597 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.538942 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.538947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.538949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.539146 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:26.539264 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:26.539721 (+ 457us) server_negotiation.cc:299] Negotiation successful
0927 10:48:26.539857 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:48:26.540880 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.528315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43097 (local address 127.9.39.129:37477)
0927 10:48:26.528767 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:26.528806 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:26.529972 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:26.530547 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:26.530554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:26.531031 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:26.531560 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:26.531578 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.532348 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.532352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:26.533480 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:26.533493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.534014 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.534019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:26.534221 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.534862 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:26.534877 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:26.536246 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:26.538111 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:26.538117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:26.538130 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:26.538374 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:26.538710 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:26.538716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:26.538718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:26.538846 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:26.539262 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:26.539269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:26.539493 (+ 224us) client_negotiation.cc:769] Sending connection context
0927 10:48:26.539670 (+ 177us) client_negotiation.cc:240] Negotiation successful
0927 10:48:26.540004 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:48:26.542303 9617 heartbeater.cc:347] Connected to a master server at 127.9.39.190:43097
I20260927 10:48:26.542549 9617 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:26.543110 9617 heartbeater.cc:511] Master 127.9.39.190:43097 requested a full tablet report, sending...
I20260927 10:48:26.544759 9426 ts_manager.cc:194] Registered new tserver with Master: 39432be13e8d470a92fed13783ac9e1f (127.9.39.129:45519)
I20260927 10:48:26.546375 9426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:37477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:26.557987 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43097
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:26.665037 9622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:26.665287 9622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:26.665329 9622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:26.669320 9622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:26.669370 9622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:26.669426 9622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:26.674401 9622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43097
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:26.675453 9622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:26.676388 9622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:26.685925 9628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.685956 9630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.685966 9627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:26.686203 9622 server_base.cc:1034] running on GCE node
I20260927 10:48:26.686575 9622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:26.687248 9622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:26.688418 9622 hybrid_clock.cc:648] HybridClock initialized: now 1790506106688380 us; error 45 us; skew 500 ppm
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:26.691368 9622 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:26.692214 9622 webserver.cc:460] Webserver started at http://127.9.39.130:36825/ using document root <none> and password file <none>
I20260927 10:48:26.692778 9622 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:26.692870 9622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:26.693116 9622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:26.695199 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "d24397871cbd4a0bbbf4c7a4f8e51ba6"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "4fa2a5581847e204445e3602fa96deee"
server_key_iv: "a65e74b49c84e795c6aa93ee07c375a1"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.695750 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "d24397871cbd4a0bbbf4c7a4f8e51ba6"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "4fa2a5581847e204445e3602fa96deee"
server_key_iv: "a65e74b49c84e795c6aa93ee07c375a1"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.699496 9622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:26.701916 9637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.703207 9622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:26.703343 9622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "d24397871cbd4a0bbbf4c7a4f8e51ba6"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "4fa2a5581847e204445e3602fa96deee"
server_key_iv: "a65e74b49c84e795c6aa93ee07c375a1"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.703464 9622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:26.723824 9622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:26.726961 9622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:26.727157 9622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:26.727761 9622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:26.728545 9622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:26.728590 9622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.728631 9622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:26.728650 9622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.737087 9622 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:41273
I20260927 10:48:26.737106 9750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:41273 every 8 connection(s)
I20260927 10:48:26.738027 9622 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:26.744160 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 9622
I20260927 10:48:26.744277 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:26.744552 9374 external_mini_cluster.cc:1428] Setting key 65888f72326dc82e6e741c28d0bcf4c4
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:26.752588 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.739856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:46597 (local address 127.9.39.190:43097)
0927 10:48:26.739984 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:26.739988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:26.740778 (+ 790us) server_negotiation.cc:407] Connection header received
0927 10:48:26.742047 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:26.742050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:26.742095 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:26.742170 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:26.743297 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.743809 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.745532 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.745766 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.745903 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.748666 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:26.748669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:26.748671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:26.748694 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:26.750378 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.751067 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.751072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.751074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.751159 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.751536 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.751539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.751541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.751723 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:26.751827 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:26.752309 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 10:48:26.752426 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:48:26.753364 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.740213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43097 (local address 127.9.39.130:46597)
0927 10:48:26.740636 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:26.740674 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:26.741853 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:26.742317 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:26.742323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:26.742648 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:26.743174 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:26.743187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.743908 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.743910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:26.745378 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:26.745388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.745935 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.745938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:26.746132 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.746780 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:26.746806 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:26.748484 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:26.750610 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:26.750616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:26.750631 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:26.750960 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:26.751264 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:26.751267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:26.751269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:26.751416 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:26.751825 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:26.751832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:26.752075 (+ 243us) client_negotiation.cc:769] Sending connection context
0927 10:48:26.752256 (+ 181us) client_negotiation.cc:240] Negotiation successful
0927 10:48:26.752587 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:48:26.754678 9751 heartbeater.cc:347] Connected to a master server at 127.9.39.190:43097
I20260927 10:48:26.754945 9751 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:26.755474 9751 heartbeater.cc:511] Master 127.9.39.190:43097 requested a full tablet report, sending...
I20260927 10:48:26.756568 9426 ts_manager.cc:194] Registered new tserver with Master: d24397871cbd4a0bbbf4c7a4f8e51ba6 (127.9.39.130:41273)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:48:26.757223 9426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:46597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:26.771718 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43097
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:26.880880 9756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:26.881127 9756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:26.881168 9756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:26.885300 9756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:26.885354 9756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:26.885440 9756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:26.890906 9756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43097
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:26.891947 9756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:26.892925 9756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:48:26.902562 9756 server_base.cc:1034] running on GCE node
W20260927 10:48:26.902499 9764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.902642 9762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:26.902499 9761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:26.903194 9756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:26.903842 9756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:26.904999 9756 hybrid_clock.cc:648] HybridClock initialized: now 1790506106904983 us; error 28 us; skew 500 ppm
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:26.907840 9756 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:26.908650 9756 webserver.cc:460] Webserver started at http://127.9.39.131:41909/ using document root <none> and password file <none>
I20260927 10:48:26.909197 9756 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:26.909282 9756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:26.909507 9756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:26.911550 9756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "7a3045708d414f8fbdb99760b7aaa2c6"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "12d30f9d17f39f02f3b159f4d5b0b391"
server_key_iv: "dd2f6ff66b0629e78bbea4b3ee79b6a7"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.912109 9756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "7a3045708d414f8fbdb99760b7aaa2c6"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "12d30f9d17f39f02f3b159f4d5b0b391"
server_key_iv: "dd2f6ff66b0629e78bbea4b3ee79b6a7"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.915789 9756 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:26.918180 9771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.919566 9756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:26.919708 9756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "7a3045708d414f8fbdb99760b7aaa2c6"
format_stamp: "Formatted at 2026-09-27 10:48:26 on dist-test-slave-p9q6"
server_key: "12d30f9d17f39f02f3b159f4d5b0b391"
server_key_iv: "dd2f6ff66b0629e78bbea4b3ee79b6a7"
server_key_version: "encryptionkey@0"
I20260927 10:48:26.919813 9756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:26.935631 9756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:26.938418 9756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:26.938623 9756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:26.939208 9756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:26.940038 9756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:26.940109 9756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.940179 9756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:26.940224 9756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:26.949203 9756 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:35853
I20260927 10:48:26.949221 9884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:35853 every 8 connection(s)
I20260927 10:48:26.950078 9756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:26.957561 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 9756
I20260927 10:48:26.957684 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:26.957950 9374 external_mini_cluster.cc:1428] Setting key 38f925b73dd9b528d99b73deff9a99bb
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:26.963456 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.951873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34743 (local address 127.9.39.190:43097)
0927 10:48:26.952009 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:26.952013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:26.952810 (+ 797us) server_negotiation.cc:407] Connection header received
0927 10:48:26.954139 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:26.954143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:26.954196 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:26.954313 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:26.955540 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.956064 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.957192 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.957416 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.957525 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.960183 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:26.960187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:26.960190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:26.960218 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:26.961692 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.962141 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.962144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.962145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.962215 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.962500 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.962502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.962503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.962670 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:26.962804 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:26.963247 (+ 443us) server_negotiation.cc:299] Negotiation successful
0927 10:48:26.963341 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:48:26.964249 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.952258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43097 (local address 127.9.39.131:34743)
0927 10:48:26.952670 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:26.952711 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:26.953925 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:26.954481 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:26.954487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:26.954915 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:26.955384 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:26.955397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.956193 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.956197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:26.957057 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:26.957066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.957562 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.957566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:26.957818 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.958443 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:26.958458 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:26.960007 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:26.961791 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:26.961796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:26.961807 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:26.962053 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:26.962292 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:26.962294 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:26.962296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:26.962424 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:26.962815 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:26.962821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:26.963035 (+ 214us) client_negotiation.cc:769] Sending connection context
0927 10:48:26.963201 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 10:48:26.963443 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
I20260927 10:48:26.965350 9885 heartbeater.cc:347] Connected to a master server at 127.9.39.190:43097
I20260927 10:48:26.965584 9885 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:26.966044 9885 heartbeater.cc:511] Master 127.9.39.190:43097 requested a full tablet report, sending...
I20260927 10:48:26.967046 9426 ts_manager.cc:194] Registered new tserver with Master: 7a3045708d414f8fbdb99760b7aaa2c6 (127.9.39.131:35853)
I20260927 10:48:26.967598 9426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:34743
I20260927 10:48:26.972154 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 10:48:26 dist-test-slave-p9q6 krb5kdc[9381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506106, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:26.997606 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:26.988662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.9.39.129:45519)
0927 10:48:26.988995 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:26.989007 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:26.989060 (+ 53us) server_negotiation.cc:407] Connection header received
0927 10:48:26.989159 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:26.989164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:26.989354 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:26.989466 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:26.990202 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.991058 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.991832 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:26.992179 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:26.992257 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:26.994526 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:26.994534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:26.994564 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:26.994586 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:26.996097 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.996459 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.996464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.996465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.996559 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:26.996786 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:26.996788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:26.996789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:26.996991 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:26.997139 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:26.997303 (+ 164us) server_negotiation.cc:299] Negotiation successful
0927 10:48:26.997431 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
W20260927 10:48:26.998528 9531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518
Sep 27 10:48:27 dist-test-slave-p9q6 krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 10:48:27 dist-test-slave-p9q6 krb5kdc[9381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:27.023289 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:27.015553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.9.39.129:45519)
0927 10:48:27.015684 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:27.015688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:27.015895 (+ 207us) server_negotiation.cc:407] Connection header received
0927 10:48:27.015964 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:27.015966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:27.016006 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:27.016080 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:27.016691 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:27.017280 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:27.018103 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:27.018335 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:27.018430 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:27.020647 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:27.020651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:27.020653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:27.020673 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:27.021887 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:27.022238 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:27.022240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:27.022242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:27.022308 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:27.022517 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:27.022519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:27.022520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:27.022667 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:27.022826 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:27.022990 (+ 164us) server_negotiation.cc:299] Negotiation successful
0927 10:48:27.023121 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:48:27.024340 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 9488
I20260927 10:48:27.030736 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 9622
I20260927 10:48:27.035527 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 9756
I20260927 10:48:27.040498 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 9394
2026-09-27T10:48:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2132 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:27 dist-test-slave-p9q6 krb5kdc[9899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:27 dist-test-slave-p9q6 krb5kdc[9899](info): set up 2 sockets
Sep 27 10:48:27 dist-test-slave-p9q6 krb5kdc[9899](info): commencing operation
krb5kdc: starting...
W20260927 10:48:28.070293 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:28.189692 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37365
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:37365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:28.294359 9912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:28.294587 9912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:28.294624 9912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:28.298431 9912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:28.298480 9912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:28.298498 9912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:28.298513 9912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:28.298528 9912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:28.303422 9912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:37365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:28.304296 9912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:28.305167 9912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:28.313369 9918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:28.313427 9912 server_base.cc:1034] running on GCE node
W20260927 10:48:28.313341 9920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:28.313419 9917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:28.314199 9912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:28.315294 9912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:28.316502 9912 hybrid_clock.cc:648] HybridClock initialized: now 1790506108316462 us; error 49 us; skew 500 ppm
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:28.319270 9912 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:28.320291 9912 webserver.cc:460] Webserver started at http://127.9.39.190:42069/ using document root <none> and password file <none>
I20260927 10:48:28.320842 9912 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:28.320947 9912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:28.321167 9912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:28.323236 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "5b560c6d3c7540d1b2e177680a4cf479"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "0de89ffa3a217e961c3fde8f04cce999"
server_key_iv: "95199aec142f2cda13b1439bbf7b2005"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.323833 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "5b560c6d3c7540d1b2e177680a4cf479"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "0de89ffa3a217e961c3fde8f04cce999"
server_key_iv: "95199aec142f2cda13b1439bbf7b2005"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.327522 9912 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:48:28.329788 9927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.330988 9912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:28.331125 9912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "5b560c6d3c7540d1b2e177680a4cf479"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "0de89ffa3a217e961c3fde8f04cce999"
server_key_iv: "95199aec142f2cda13b1439bbf7b2005"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.331238 9912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:28.345193 9912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:28.348815 9912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:28.349042 9912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:28.357870 9912 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:37365
I20260927 10:48:28.357882 9979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:37365 every 8 connection(s)
I20260927 10:48:28.358860 9912 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:28.361464 9980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:28.365115 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 9912
I20260927 10:48:28.365206 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:28.365476 9374 external_mini_cluster.cc:1428] Setting key 27c2b5d0100b54bc3615f4a52ee6c3b3
I20260927 10:48:28.366942 9980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479: Bootstrap starting.
I20260927 10:48:28.369561 9980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:28.370416 9980 log.cc:824] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:28.372311 9980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479: No bootstrap required, opened a new log
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:28.376251 9980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b560c6d3c7540d1b2e177680a4cf479" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37365 } }
I20260927 10:48:28.376523 9980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:28.376590 9980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b560c6d3c7540d1b2e177680a4cf479, State: Initialized, Role: FOLLOWER
I20260927 10:48:28.377054 9980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b560c6d3c7540d1b2e177680a4cf479" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37365 } }
I20260927 10:48:28.377208 9980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:28.377269 9980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:28.377353 9980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:28.378518 9980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b560c6d3c7540d1b2e177680a4cf479" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37365 } }
I20260927 10:48:28.378875 9980 leader_election.cc:302] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b560c6d3c7540d1b2e177680a4cf479; no voters:
I20260927 10:48:28.379199 9980 leader_election.cc:288] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:28.379274 9985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:28.379464 9985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [term 1 LEADER]: Becoming Leader. State: Replica: 5b560c6d3c7540d1b2e177680a4cf479, State: Running, Role: LEADER
I20260927 10:48:28.379736 9985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b560c6d3c7540d1b2e177680a4cf479" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37365 } }
I20260927 10:48:28.380429 9980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:28.380825 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:28.366914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.9.39.190:37365)
0927 10:48:28.367350 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:28.367368 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:28.367407 (+ 39us) server_negotiation.cc:407] Connection header received
0927 10:48:28.368240 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:28.368265 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:28.368662 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:28.369016 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:28.369667 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.371054 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.371971 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.372621 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.372784 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:28.375372 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:28.375378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:28.375454 (+ 76us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:28.375490 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:28.377788 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:28.378252 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:28.378257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:28.378260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:28.378364 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:28.378671 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:28.378674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:28.378676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:28.379146 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:28.379417 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:28.379825 (+ 408us) server_negotiation.cc:299] Negotiation successful
0927 10:48:28.380076 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":156,"threads_started":1}
I20260927 10:48:28.380839 9985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b560c6d3c7540d1b2e177680a4cf479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b560c6d3c7540d1b2e177680a4cf479" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37365 } } }
I20260927 10:48:28.380975 9985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:28.381249 9986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b560c6d3c7540d1b2e177680a4cf479. Latest consensus state: current_term: 1 leader_uuid: "5b560c6d3c7540d1b2e177680a4cf479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b560c6d3c7540d1b2e177680a4cf479" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37365 } } }
I20260927 10:48:28.381348 9989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:28.381364 9986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:28.384809 9989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:28.390293 9989 catalog_manager.cc:1347] Generated new cluster ID: 106ba624c9a54dd59835aba6dd548057
I20260927 10:48:28.390373 9989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:28.421418 9989 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:28.422761 9989 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:28.433203 9989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b560c6d3c7540d1b2e177680a4cf479: Generated new TSK 0
I20260927 10:48:28.434024 9989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:28.467993 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37365
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:28.576191 10006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:28.576455 10006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:28.576498 10006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:28.580509 10006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:28.580569 10006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:28.580629 10006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:28.586097 10006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37365
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:28.587154 10006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:28.588126 10006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:28.597383 10014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:28.597383 10012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:28.597730 10011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:28.597900 10006 server_base.cc:1034] running on GCE node
I20260927 10:48:28.598338 10006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:28.599018 10006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:28.600174 10006 hybrid_clock.cc:648] HybridClock initialized: now 1790506108600159 us; error 29 us; skew 500 ppm
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:28.603084 10006 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:28.603940 10006 webserver.cc:460] Webserver started at http://127.9.39.129:44939/ using document root <none> and password file <none>
I20260927 10:48:28.604439 10006 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:28.604496 10006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:28.604650 10006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:28.606555 10006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "f667417798184ceba2bb28b979a7e404"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "51c10e8ac3ceeeec83a6b11b830de112"
server_key_iv: "dc37b4d28b5848d0f731c1409f872cc2"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.607161 10006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "f667417798184ceba2bb28b979a7e404"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "51c10e8ac3ceeeec83a6b11b830de112"
server_key_iv: "dc37b4d28b5848d0f731c1409f872cc2"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.610633 10006 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:28.613003 10021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.614210 10006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 10:48:28.614300 10006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "f667417798184ceba2bb28b979a7e404"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "51c10e8ac3ceeeec83a6b11b830de112"
server_key_iv: "dc37b4d28b5848d0f731c1409f872cc2"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.614380 10006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:28.638108 10006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:28.641573 10006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:28.641731 10006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:28.642279 10006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:28.643143 10006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:28.643189 10006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.643229 10006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:28.643254 10006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.652450 10006 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:34613
I20260927 10:48:28.652474 10134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:34613 every 8 connection(s)
I20260927 10:48:28.653357 10006 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:28.654116 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10006
I20260927 10:48:28.654203 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:28.654465 9374 external_mini_cluster.cc:1428] Setting key 7beb24a0e9e4c4c6a98c9b31a927cb38
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:48:28.667359 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:28.655239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:36075 (local address 127.9.39.190:37365)
0927 10:48:28.655342 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:28.655347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:28.656499 (+ 1152us) server_negotiation.cc:407] Connection header received
0927 10:48:28.657567 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:28.657571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:28.657623 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:28.657731 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:28.658967 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.659509 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.660659 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.660887 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.661003 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:28.663732 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:28.663735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:28.663737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:28.663760 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:28.665252 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:28.665776 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:28.665781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:28.665783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:28.665903 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:28.666245 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:28.666249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:28.666250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:28.666427 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:28.666535 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:28.667063 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 10:48:28.667191 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 10:48:28.668138 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:28.655769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37365 (local address 127.9.39.129:36075)
0927 10:48:28.656298 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:28.656364 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:28.657360 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:28.657910 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:28.657917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:28.658258 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:28.658795 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:28.658819 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.659634 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.659638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:28.660531 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:28.660540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.661040 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.661044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:28.661221 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:28.661841 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:28.661856 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:28.663563 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:28.665376 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:28.665411 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:28.665432 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:28.665670 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:28.666003 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:28.666007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:28.666009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:28.666141 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:28.666532 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:28.666537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:28.666834 (+ 297us) client_negotiation.cc:769] Sending connection context
0927 10:48:28.667021 (+ 187us) client_negotiation.cc:240] Negotiation successful
0927 10:48:28.667340 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
I20260927 10:48:28.669447 10135 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37365
I20260927 10:48:28.669727 10135 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:28.670225 10135 heartbeater.cc:511] Master 127.9.39.190:37365 requested a full tablet report, sending...
I20260927 10:48:28.672016 9936 ts_manager.cc:194] Registered new tserver with Master: f667417798184ceba2bb28b979a7e404 (127.9.39.129:34613)
I20260927 10:48:28.673280 9936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:36075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:28.682101 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37365
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:28.793783 10140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:28.794040 10140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:28.794085 10140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:28.798056 10140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:28.798115 10140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:28.798177 10140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:28.803223 10140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37365
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:28.804195 10140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:28.805154 10140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:28.814640 10148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:28.814649 10146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:28.814656 10145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:28.814858 10140 server_base.cc:1034] running on GCE node
I20260927 10:48:28.815379 10140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:28.816032 10140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:28.817216 10140 hybrid_clock.cc:648] HybridClock initialized: now 1790506108817181 us; error 47 us; skew 500 ppm
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:28.819981 10140 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:28.820863 10140 webserver.cc:460] Webserver started at http://127.9.39.130:40411/ using document root <none> and password file <none>
I20260927 10:48:28.821542 10140 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:28.821631 10140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:28.821837 10140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:28.823962 10140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "3fd19a06d47e47658e271a386300ecaf"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "29dcdf30a2de9851b7388ce028b0642f"
server_key_iv: "fa8d1825f6923e208b7821e5ada10464"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.824509 10140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "3fd19a06d47e47658e271a386300ecaf"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "29dcdf30a2de9851b7388ce028b0642f"
server_key_iv: "fa8d1825f6923e208b7821e5ada10464"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.828179 10140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:28.830668 10155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.832003 10140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:28.832151 10140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3fd19a06d47e47658e271a386300ecaf"
format_stamp: "Formatted at 2026-09-27 10:48:28 on dist-test-slave-p9q6"
server_key: "29dcdf30a2de9851b7388ce028b0642f"
server_key_iv: "fa8d1825f6923e208b7821e5ada10464"
server_key_version: "encryptionkey@0"
I20260927 10:48:28.832265 10140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:28.852535 10140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:28.855404 10140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:28.855605 10140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:28.856222 10140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:28.857019 10140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:28.857089 10140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.857160 10140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:28.857203 10140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:28.866374 10140 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:43115
I20260927 10:48:28.866420 10268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:43115 every 8 connection(s)
I20260927 10:48:28.867331 10140 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:28.868299 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10140
I20260927 10:48:28.868420 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:28.868702 9374 external_mini_cluster.cc:1428] Setting key 03f6f51a88f4b27b9d12a6ca029a4e05
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Sep 27 10:48:28 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:48:28.882921 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:28.869426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:35151 (local address 127.9.39.190:37365)
0927 10:48:28.869566 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:28.869571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:28.870508 (+ 937us) server_negotiation.cc:407] Connection header received
0927 10:48:28.872412 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:28.872419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:28.872480 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:28.872584 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:28.873809 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.874744 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.875909 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.876292 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.876408 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:28.878877 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:28.878880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:28.878882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:28.878905 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:28.880323 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:28.880985 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:28.880991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:28.880996 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:28.881096 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:28.881645 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:28.881650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:28.881652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:28.881826 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:28.881953 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:28.882618 (+ 665us) server_negotiation.cc:299] Negotiation successful
0927 10:48:28.882792 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:48:28.884141 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:28.869788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37365 (local address 127.9.39.130:35151)
0927 10:48:28.870330 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:28.870398 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:28.872171 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:28.872732 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:28.872739 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:28.873094 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:28.873650 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:28.873663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.874860 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.874863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:28.875773 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:28.875783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:28.876458 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:28.876461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:28.876610 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:28.877217 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:28.877232 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:28.878670 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:28.880506 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:28.880510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:28.880521 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:28.880844 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:28.881210 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:28.881214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:28.881216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:28.881529 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:28.881934 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:28.881944 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:28.882256 (+ 312us) client_negotiation.cc:769] Sending connection context
0927 10:48:28.883114 (+ 858us) client_negotiation.cc:240] Negotiation successful
0927 10:48:28.883405 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":145,"threads_started":1}
I20260927 10:48:28.885339 10269 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37365
I20260927 10:48:28.885607 10269 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:28.886086 10269 heartbeater.cc:511] Master 127.9.39.190:37365 requested a full tablet report, sending...
I20260927 10:48:28.887166 9936 ts_manager.cc:194] Registered new tserver with Master: 3fd19a06d47e47658e271a386300ecaf (127.9.39.130:43115)
I20260927 10:48:28.887977 9936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:35151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:28.897450 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37365
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:29.006584 10274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:29.006913 10274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:29.006987 10274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:29.010973 10274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:29.011053 10274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:29.011152 10274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:29.016222 10274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37365
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:29.017182 10274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:29.018174 10274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:29.027704 10279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:29.027704 10282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:29.027827 10274 server_base.cc:1034] running on GCE node
W20260927 10:48:29.027743 10280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:29.028347 10274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:29.028962 10274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:29.030124 10274 hybrid_clock.cc:648] HybridClock initialized: now 1790506109030112 us; error 24 us; skew 500 ppm
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:29.032940 10274 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:29.033849 10274 webserver.cc:460] Webserver started at http://127.9.39.131:35795/ using document root <none> and password file <none>
I20260927 10:48:29.034462 10274 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:29.034562 10274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:29.034896 10274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:29.036945 10274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db"
format_stamp: "Formatted at 2026-09-27 10:48:29 on dist-test-slave-p9q6"
server_key: "c2772aa296b8f53ba1932ddcb4914f7b"
server_key_iv: "c65c6abe29d3ba81b1a02f002617cc9b"
server_key_version: "encryptionkey@0"
I20260927 10:48:29.037554 10274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db"
format_stamp: "Formatted at 2026-09-27 10:48:29 on dist-test-slave-p9q6"
server_key: "c2772aa296b8f53ba1932ddcb4914f7b"
server_key_iv: "c65c6abe29d3ba81b1a02f002617cc9b"
server_key_version: "encryptionkey@0"
I20260927 10:48:29.041239 10274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:48:29.043695 10289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:29.044811 10274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:29.044960 10274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db"
format_stamp: "Formatted at 2026-09-27 10:48:29 on dist-test-slave-p9q6"
server_key: "c2772aa296b8f53ba1932ddcb4914f7b"
server_key_iv: "c65c6abe29d3ba81b1a02f002617cc9b"
server_key_version: "encryptionkey@0"
I20260927 10:48:29.045071 10274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:29.064574 10274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:29.067423 10274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:29.067618 10274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:29.068171 10274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:29.068948 10274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:29.069017 10274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:29.069082 10274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:29.069133 10274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:29.077822 10274 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:33421
I20260927 10:48:29.077859 10402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:33421 every 8 connection(s)
I20260927 10:48:29.078711 10274 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:29.083142 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10274
I20260927 10:48:29.083273 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:29.083537 9374 external_mini_cluster.cc:1428] Setting key e85d0088bc92df118bb907f69ebb6551
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:29.091992 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.080426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:48341 (local address 127.9.39.190:37365)
0927 10:48:29.080562 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.080566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.081516 (+ 950us) server_negotiation.cc:407] Connection header received
0927 10:48:29.082450 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.082453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.082534 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.082638 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:29.083939 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.084506 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.086073 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.086279 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.086356 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.088746 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:29.088749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:29.088751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:29.088778 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:29.090195 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.090650 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.090652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.090653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.090787 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.091087 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.091089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.091090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.091250 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:29.091322 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.091777 (+ 455us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.091878 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:48:29.092728 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.080847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37365 (local address 127.9.39.131:48341)
0927 10:48:29.081336 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:29.081384 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:29.082250 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:29.082835 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:29.082848 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:29.083185 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:29.083761 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.083776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.084686 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.084689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.085965 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.085975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.086420 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.086423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.086563 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.087177 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:29.087193 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:29.088592 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:29.090305 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:29.090309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:29.090320 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:29.090565 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:29.090864 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:29.090867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:29.090868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:29.091007 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:29.091333 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:29.091338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:29.091548 (+ 210us) client_negotiation.cc:769] Sending connection context
0927 10:48:29.091696 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 10:48:29.091935 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":109,"threads_started":1}
I20260927 10:48:29.093811 10403 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37365
I20260927 10:48:29.094033 10403 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:29.094488 10403 heartbeater.cc:511] Master 127.9.39.190:37365 requested a full tablet report, sending...
I20260927 10:48:29.095503 9936 ts_manager.cc:194] Registered new tserver with Master: f9db5d2078d0495bb1cb2fc72cc4f7db (127.9.39.131:33421)
I20260927 10:48:29.096136 9936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:48341
I20260927 10:48:29.097347 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:48:29.108070 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.100731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.9.39.190:37365)
0927 10:48:29.100902 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.100906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.101053 (+ 147us) server_negotiation.cc:407] Connection header received
0927 10:48:29.101200 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.101203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.101250 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.101332 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:29.102007 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.102500 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.103404 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.103619 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.103726 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.104942 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:29.104945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:29.104948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:29.104976 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:29.106185 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.106662 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.106668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.106670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.106804 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.107125 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.107128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.107129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.107367 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:29.107481 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.107750 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.107846 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 10:48:29.112685 9936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:29.115654 9936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:29.129586 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.123662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43115 (local address 127.0.0.1:56648)
0927 10:48:29.124115 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:29.124158 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:29.124278 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:29.125218 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:29.125221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:29.125233 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:29.125294 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.125298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.126655 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.126658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.127974 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.127985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.129054 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.129057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.129169 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.129204 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 10:48:29.129313 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 10:48:29.129429 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":79,"thread_start_us":58,"threads_started":1}
I20260927 10:48:29.129629 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.124268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56648 (local address 127.9.39.130:43115)
0927 10:48:29.124797 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.124808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.124828 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:48:29.124896 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.124921 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.125095 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.125235 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:29.125384 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.126521 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.128118 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.128932 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.129034 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.129167 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.129359 (+ 192us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.129467 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":106,"threads_started":1}
I20260927 10:48:29.129892 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.123894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57458 (local address 127.9.39.131:33421)
0927 10:48:29.124325 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.124336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.124566 (+ 230us) server_negotiation.cc:407] Connection header received
0927 10:48:29.124648 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.124653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.124860 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.125027 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:29.125403 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.126271 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.128118 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.128928 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.129034 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.129168 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.129655 (+ 487us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.129761 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":90,"threads_started":1}
I20260927 10:48:29.129918 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.125001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.9.39.129:34613)
0927 10:48:29.125543 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.125559 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.125579 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:48:29.125664 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.125669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.125831 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.125972 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:29.126165 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.127097 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.128411 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.128945 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.129064 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.129198 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.129653 (+ 455us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.129748 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":64,"threads_started":1}
I20260927 10:48:29.130003 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.124661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:34613 (local address 127.0.0.1:48584)
0927 10:48:29.124936 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:29.124950 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:29.125021 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:29.125975 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:29.125978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:29.125997 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:29.126076 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.126081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.127222 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.127228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.128289 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.128298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.129256 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.129262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.129372 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.129423 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 10:48:29.129486 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 10:48:29.129544 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":19,"thread_start_us":78,"threads_started":1}
I20260927 10:48:29.130163 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.123764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33421 (local address 127.0.0.1:57458)
0927 10:48:29.124352 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:29.124382 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:29.124472 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:29.125133 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:29.125138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:29.125159 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:29.125284 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.125290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.126406 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.126413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.127974 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.127985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.129236 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.129239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.129332 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.129399 (+ 67us) client_negotiation.cc:769] Sending connection context
0927 10:48:29.129466 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 10:48:29.129531 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":124,"thread_start_us":111,"threads_started":1}
I20260927 10:48:29.132623 10069 tablet_service.cc:1467] Processing CreateTablet for tablet 48329c5c31fc43c6904cef2eafc317d3 (DEFAULT_TABLE table=test-table [id=ff47efb45fc34940986dab1e117879f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:29.132786 10203 tablet_service.cc:1467] Processing CreateTablet for tablet 48329c5c31fc43c6904cef2eafc317d3 (DEFAULT_TABLE table=test-table [id=ff47efb45fc34940986dab1e117879f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:29.132786 10337 tablet_service.cc:1467] Processing CreateTablet for tablet 48329c5c31fc43c6904cef2eafc317d3 (DEFAULT_TABLE table=test-table [id=ff47efb45fc34940986dab1e117879f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:29.133481 10069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48329c5c31fc43c6904cef2eafc317d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:29.133734 10203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48329c5c31fc43c6904cef2eafc317d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:29.133734 10337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48329c5c31fc43c6904cef2eafc317d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:29.140094 10420 tablet_bootstrap.cc:492] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: Bootstrap starting.
I20260927 10:48:29.141280 10421 tablet_bootstrap.cc:492] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: Bootstrap starting.
I20260927 10:48:29.142251 10420 tablet_bootstrap.cc:654] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:29.143038 10422 tablet_bootstrap.cc:492] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: Bootstrap starting.
I20260927 10:48:29.143075 10421 tablet_bootstrap.cc:654] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:29.143371 10420 log.cc:824] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:29.144264 10421 log.cc:824] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:29.145082 10422 tablet_bootstrap.cc:654] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:29.145411 10420 tablet_bootstrap.cc:492] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: No bootstrap required, opened a new log
I20260927 10:48:29.145606 10420 ts_tablet_manager.cc:1397] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:48:29.146284 10422 log.cc:824] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:29.146814 10421 tablet_bootstrap.cc:492] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: No bootstrap required, opened a new log
I20260927 10:48:29.147011 10421 ts_tablet_manager.cc:1397] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:29.148294 10422 tablet_bootstrap.cc:492] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: No bootstrap required, opened a new log
I20260927 10:48:29.148523 10422 ts_tablet_manager.cc:1397] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:48:29.149003 10420 raft_consensus.cc:348] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.149291 10420 raft_consensus.cc:374] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:29.149344 10420 raft_consensus.cc:729] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f667417798184ceba2bb28b979a7e404, State: Initialized, Role: FOLLOWER
I20260927 10:48:29.149813 10420 consensus_queue.cc:260] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.150434 10135 heartbeater.cc:503] Master 127.9.39.190:37365 was elected leader, sending a full tablet report...
I20260927 10:48:29.150789 10420 ts_tablet_manager.cc:1428] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:48:29.150949 10421 raft_consensus.cc:348] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.151201 10421 raft_consensus.cc:374] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:29.151259 10421 raft_consensus.cc:729] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd19a06d47e47658e271a386300ecaf, State: Initialized, Role: FOLLOWER
I20260927 10:48:29.151860 10421 consensus_queue.cc:260] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.152254 10422 raft_consensus.cc:348] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.152505 10422 raft_consensus.cc:374] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:29.152506 10269 heartbeater.cc:503] Master 127.9.39.190:37365 was elected leader, sending a full tablet report...
I20260927 10:48:29.152556 10422 raft_consensus.cc:729] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9db5d2078d0495bb1cb2fc72cc4f7db, State: Initialized, Role: FOLLOWER
I20260927 10:48:29.152812 10421 ts_tablet_manager.cc:1428] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:29.152963 10422 consensus_queue.cc:260] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.153541 10403 heartbeater.cc:503] Master 127.9.39.190:37365 was elected leader, sending a full tablet report...
I20260927 10:48:29.153855 10422 ts_tablet_manager.cc:1428] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260927 10:48:29.154395 10136 tablet.cc:2369] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:29.323462 10428 raft_consensus.cc:482] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:29.323716 10428 raft_consensus.cc:504] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.324858 10428 leader_election.cc:288] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fd19a06d47e47658e271a386300ecaf (127.9.39.130:43115), f667417798184ceba2bb28b979a7e404 (127.9.39.129:34613)
I20260927 10:48:29.328845 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.325169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:36053 (local address 127.9.39.129:34613)
0927 10:48:29.325321 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.325326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.325377 (+ 51us) server_negotiation.cc:407] Connection header received
0927 10:48:29.325539 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.325543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.325608 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.325695 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:29.326055 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.326652 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.327818 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.328209 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.328301 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.328347 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.328636 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.328706 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 10:48:29.328857 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.325197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:59137 (local address 127.9.39.130:43115)
0927 10:48:29.325312 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.325318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.325719 (+ 401us) server_negotiation.cc:407] Connection header received
0927 10:48:29.325790 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.325793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.325845 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.325947 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:29.326234 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.326821 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.327870 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.328296 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.328438 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.328498 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.328665 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.328736 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 10:48:29.328858 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.325103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:34613 (local address 127.9.39.131:36053)
0927 10:48:29.325311 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:29.325325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:29.325421 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:29.325750 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:29.325754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:29.325777 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:29.325930 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.325940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.326824 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.326829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.327686 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.327695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.328328 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.328334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.328489 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.328520 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 10:48:29.328595 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 10:48:29.328689 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":24}
I20260927 10:48:29.328871 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.325103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43115 (local address 127.9.39.131:59137)
0927 10:48:29.325539 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:29.325552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:29.325680 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:29.325962 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:29.325964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:29.325978 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:29.326120 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.326127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.326938 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.326941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.327716 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:29.327724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.328429 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.328432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:29.328529 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.328546 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 10:48:29.328638 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 10:48:29.328706 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":47,"thread_start_us":111,"threads_started":1}
I20260927 10:48:29.329561 10089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48329c5c31fc43c6904cef2eafc317d3" candidate_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f667417798184ceba2bb28b979a7e404" is_pre_election: true
W20260927 10:48:29.329627 10404 tablet.cc:2369] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:29.329732 10223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48329c5c31fc43c6904cef2eafc317d3" candidate_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd19a06d47e47658e271a386300ecaf" is_pre_election: true
I20260927 10:48:29.329908 10089 raft_consensus.cc:2454] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9db5d2078d0495bb1cb2fc72cc4f7db in term 0.
I20260927 10:48:29.330010 10223 raft_consensus.cc:2454] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9db5d2078d0495bb1cb2fc72cc4f7db in term 0.
I20260927 10:48:29.330470 10290 leader_election.cc:302] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd19a06d47e47658e271a386300ecaf, f9db5d2078d0495bb1cb2fc72cc4f7db; no voters:
I20260927 10:48:29.330789 10428 raft_consensus.cc:2781] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:29.330894 10428 raft_consensus.cc:482] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:29.330963 10428 raft_consensus.cc:3037] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:29.332015 10428 raft_consensus.cc:504] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.332350 10428 leader_election.cc:288] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [CANDIDATE]: Term 1 election: Requested vote from peers 3fd19a06d47e47658e271a386300ecaf (127.9.39.130:43115), f667417798184ceba2bb28b979a7e404 (127.9.39.129:34613)
I20260927 10:48:29.332723 10223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48329c5c31fc43c6904cef2eafc317d3" candidate_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd19a06d47e47658e271a386300ecaf"
I20260927 10:48:29.332847 10223 raft_consensus.cc:3037] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:29.332895 10089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48329c5c31fc43c6904cef2eafc317d3" candidate_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f667417798184ceba2bb28b979a7e404"
I20260927 10:48:29.333019 10089 raft_consensus.cc:3037] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:29.333902 10223 raft_consensus.cc:2454] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9db5d2078d0495bb1cb2fc72cc4f7db in term 1.
I20260927 10:48:29.334100 10089 raft_consensus.cc:2454] T 48329c5c31fc43c6904cef2eafc317d3 P f667417798184ceba2bb28b979a7e404 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9db5d2078d0495bb1cb2fc72cc4f7db in term 1.
I20260927 10:48:29.334209 10290 leader_election.cc:302] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd19a06d47e47658e271a386300ecaf, f9db5d2078d0495bb1cb2fc72cc4f7db; no voters:
I20260927 10:48:29.334384 10428 raft_consensus.cc:2781] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:29.334640 10428 raft_consensus.cc:686] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [term 1 LEADER]: Becoming Leader. State: Replica: f9db5d2078d0495bb1cb2fc72cc4f7db, State: Running, Role: LEADER
I20260927 10:48:29.334978 10428 consensus_queue.cc:237] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } }
I20260927 10:48:29.338253 9936 catalog_manager.cc:5495] T 48329c5c31fc43c6904cef2eafc317d3 P f9db5d2078d0495bb1cb2fc72cc4f7db reported cstate change: term changed from 0 to 1, leader changed from <none> to f9db5d2078d0495bb1cb2fc72cc4f7db (127.9.39.131). New cstate: current_term: 1 leader_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd19a06d47e47658e271a386300ecaf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9db5d2078d0495bb1cb2fc72cc4f7db" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f667417798184ceba2bb28b979a7e404" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34613 } health_report { overall_health: UNKNOWN } } }
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:29.366617 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.357680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.9.39.129:34613)
0927 10:48:29.357830 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.357835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.358045 (+ 210us) server_negotiation.cc:407] Connection header received
0927 10:48:29.358192 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.358195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.358246 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.358333 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:29.359109 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.359605 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.360441 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.360661 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.360773 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.363291 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:29.363303 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:29.363316 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:29.363345 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:29.364941 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.365373 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.365378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.365379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.365456 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.365746 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.365751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.365753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.365977 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:29.366179 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.366337 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.366459 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260927 10:48:29.368471 10270 tablet.cc:2369] T 48329c5c31fc43c6904cef2eafc317d3 P 3fd19a06d47e47658e271a386300ecaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:29.391855 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.383949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.9.39.129:34613)
0927 10:48:29.384101 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.384106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.384235 (+ 129us) server_negotiation.cc:407] Connection header received
0927 10:48:29.384408 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.384411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.384458 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.384541 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:29.385188 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.385704 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.386503 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.386807 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.386915 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.389084 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:29.389088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:29.389090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:29.389113 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:29.390262 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.390649 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.390654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.390656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.390749 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.391032 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.391035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.391037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.391189 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:29.391319 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.391578 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.391690 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[9899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:29.415241 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:29.407812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.9.39.129:34613)
0927 10:48:29.407964 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:29.407968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:29.408051 (+ 83us) server_negotiation.cc:407] Connection header received
0927 10:48:29.408147 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:29.408149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:29.408187 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:29.408254 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:29.408847 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.409293 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.410065 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:29.410260 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:29.410335 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:29.412497 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:29.412500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:29.412502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:29.412522 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:29.413885 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.414219 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.414221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.414222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.414287 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:29.414496 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:29.414498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:29.414499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:29.414635 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:29.414784 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:29.414951 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 10:48:29.415032 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260927 10:48:29.416507 10049 tablet_service.cc:2908] Rejecting scan request for tablet 48329c5c31fc43c6904cef2eafc317d3: Uninitialized: clean time has not yet been initialized
I20260927 10:48:29.418296 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10006
I20260927 10:48:29.424088 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10140
I20260927 10:48:29.429297 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10274
I20260927 10:48:29.434547 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 9912
2026-09-27T10:48:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2393 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[10444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[10444](info): set up 2 sockets
Sep 27 10:48:29 dist-test-slave-p9q6 krb5kdc[10444](info): commencing operation
krb5kdc: starting...
W20260927 10:48:30.470531 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:30 dist-test-slave-p9q6 krb5kdc[10444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:30.589851 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37509
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:37509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:30.697482 10457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:30.697745 10457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:30.697786 10457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:30.701655 10457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:30.701713 10457 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 10:48:30.701733 10457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:30.701754 10457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:30.701773 10457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:30.701790 10457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:30.706982 10457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:37509
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:30.707926 10457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:30.708853 10457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:30.717012 10462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:30.717013 10463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:30.716992 10465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:30.717280 10457 server_base.cc:1034] running on GCE node
I20260927 10:48:30.717926 10457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:30.719063 10457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:30.720240 10457 hybrid_clock.cc:648] HybridClock initialized: now 1790506110720197 us; error 53 us; skew 500 ppm
Sep 27 10:48:30 dist-test-slave-p9q6 krb5kdc[10444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:30.723006 10457 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:30.723868 10457 webserver.cc:460] Webserver started at http://127.9.39.190:34737/ using document root <none> and password file <none>
I20260927 10:48:30.724452 10457 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:30.724550 10457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:30.725091 10457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:30.727227 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37"
format_stamp: "Formatted at 2026-09-27 10:48:30 on dist-test-slave-p9q6"
server_key: "53fdd701a125301b6ea6e0477f8f569d"
server_key_iv: "a0b8387f8fe3cc2bd1c66085914bac06"
server_key_version: "encryptionkey@0"
I20260927 10:48:30.727784 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37"
format_stamp: "Formatted at 2026-09-27 10:48:30 on dist-test-slave-p9q6"
server_key: "53fdd701a125301b6ea6e0477f8f569d"
server_key_iv: "a0b8387f8fe3cc2bd1c66085914bac06"
server_key_version: "encryptionkey@0"
I20260927 10:48:30.731241 10457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:30.733480 10472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:30.734673 10457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:30.734854 10457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37"
format_stamp: "Formatted at 2026-09-27 10:48:30 on dist-test-slave-p9q6"
server_key: "53fdd701a125301b6ea6e0477f8f569d"
server_key_iv: "a0b8387f8fe3cc2bd1c66085914bac06"
server_key_version: "encryptionkey@0"
I20260927 10:48:30.734995 10457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:30.749023 10457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:30.752022 10457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:30.752197 10457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:30.759995 10457 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:37509
I20260927 10:48:30.760010 10534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:37509 every 8 connection(s)
I20260927 10:48:30.760887 10457 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:30.763613 10535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:30.765411 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10457
I20260927 10:48:30.765502 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:30.765743 9374 external_mini_cluster.cc:1428] Setting key 79d7fd2b8b0f1a31448cca6d55a57cb7
I20260927 10:48:30.770102 10535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: Bootstrap starting.
I20260927 10:48:30.772636 10535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:30.773356 10535 log.cc:824] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: Log is configured to *not* fsync() on all Append() calls
Sep 27 10:48:30 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:30.775462 10535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: No bootstrap required, opened a new log
I20260927 10:48:30.779870 10535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37509 } }
I20260927 10:48:30.780153 10535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:30.780225 10535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d1cb625d7cd4e87ace7fe307c1b7d37, State: Initialized, Role: FOLLOWER
I20260927 10:48:30.780707 10535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37509 } }
I20260927 10:48:30.780867 10535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:30.780937 10535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:30.781030 10535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:30.781131 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:30.767116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.9.39.190:37509)
0927 10:48:30.767509 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:30.767527 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:30.767567 (+ 40us) server_negotiation.cc:407] Connection header received
0927 10:48:30.768297 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:30.768321 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:30.768732 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:30.769087 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:30.769684 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:30.771086 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:30.771906 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:30.772582 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:30.772758 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:30.775458 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:30.775463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:30.775480 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:30.775512 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:30.778124 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:30.778558 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:30.778565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:30.778571 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:30.778753 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:30.779013 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:30.779016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:30.779018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:30.779524 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:30.779744 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:30.779901 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 10:48:30.780194 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":103,"threads_started":1}
I20260927 10:48:30.781983 10535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37509 } }
I20260927 10:48:30.782320 10535 leader_election.cc:302] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d1cb625d7cd4e87ace7fe307c1b7d37; no voters:
I20260927 10:48:30.782615 10535 leader_election.cc:288] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:30.782727 10540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:30.782933 10540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [term 1 LEADER]: Becoming Leader. State: Replica: 7d1cb625d7cd4e87ace7fe307c1b7d37, State: Running, Role: LEADER
I20260927 10:48:30.783210 10540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37509 } }
I20260927 10:48:30.783831 10535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:30.784297 10542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d1cb625d7cd4e87ace7fe307c1b7d37. Latest consensus state: current_term: 1 leader_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37509 } } }
I20260927 10:48:30.784431 10542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:30.784720 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1cb625d7cd4e87ace7fe307c1b7d37" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37509 } } }
I20260927 10:48:30.784807 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:30.784852 10544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:30.787456 10544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:30.792877 10544 catalog_manager.cc:1347] Generated new cluster ID: ea1fad67f303442d995ae94f07c5a5f9
I20260927 10:48:30.792945 10544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:30.810307 10544 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:30.811228 10544 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:30.828449 10544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: Generated new TSK 0
I20260927 10:48:30.829183 10544 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:30.865084 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37509
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:30.970093 10561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:30.970327 10561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:30.970372 10561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:30.974215 10561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:30.974260 10561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:30.974295 10561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:48:30.974329 10561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:30.979116 10561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:37509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:30.980042 10561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:30.980940 10561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:30.990248 10567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:30.990372 10566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:30.990456 10561 server_base.cc:1034] running on GCE node
W20260927 10:48:30.990236 10569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:30.990916 10561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:30.991554 10561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:30.992708 10561 hybrid_clock.cc:648] HybridClock initialized: now 1790506110992691 us; error 28 us; skew 500 ppm
Sep 27 10:48:30 dist-test-slave-p9q6 krb5kdc[10444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:30.995304 10561 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:30.996121 10561 webserver.cc:460] Webserver started at http://127.9.39.129:39039/ using document root <none> and password file <none>
I20260927 10:48:30.996647 10561 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:30.996735 10561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:30.996973 10561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:30.998960 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "549a6de2744f45af826324208435cffb"
format_stamp: "Formatted at 2026-09-27 10:48:30 on dist-test-slave-p9q6"
server_key: "336eb9b050812a7216d134d8200c630d"
server_key_iv: "1a629f9ceaf5a7365ddaded002d3de5d"
server_key_version: "encryptionkey@0"
I20260927 10:48:30.999492 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "549a6de2744f45af826324208435cffb"
format_stamp: "Formatted at 2026-09-27 10:48:30 on dist-test-slave-p9q6"
server_key: "336eb9b050812a7216d134d8200c630d"
server_key_iv: "1a629f9ceaf5a7365ddaded002d3de5d"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.002897 10561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:31.005252 10576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:31.006422 10561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:31.006556 10561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "549a6de2744f45af826324208435cffb"
format_stamp: "Formatted at 2026-09-27 10:48:30 on dist-test-slave-p9q6"
server_key: "336eb9b050812a7216d134d8200c630d"
server_key_iv: "1a629f9ceaf5a7365ddaded002d3de5d"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.006660 10561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:31.019690 10561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:31.022676 10561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:31.022907 10561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:31.024287 10561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:31.024356 10561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:31.024417 10561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:31.024467 10561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:31.034958 10561 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:39953
I20260927 10:48:31.035022 10691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:39953 every 8 connection(s)
I20260927 10:48:31.036114 10561 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:31.038529 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.025130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:35819 (local address 127.9.39.190:37509)
0927 10:48:31.025248 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.025252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.026251 (+ 999us) server_negotiation.cc:407] Connection header received
0927 10:48:31.027281 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.027285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.027339 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.027395 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.028943 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.029869 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.031305 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.031550 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.031664 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.034788 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.034792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.034795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.034824 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.036477 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.037069 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.037073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.037074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.037167 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.037545 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.037548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.037549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.037712 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.037789 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.038272 (+ 483us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.038377 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:48:31.039410 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.025644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.9.39.129:35819)
0927 10:48:31.026111 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.026149 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.027045 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.027794 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.027811 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.028194 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.028756 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.028773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.030019 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.030025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.031185 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.031194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.031705 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.031709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.031868 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.033017 (+ 1149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.033036 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.034596 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.036601 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.036606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.036619 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.036943 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.037322 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.037324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.037325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.037457 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.037811 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.037815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.038062 (+ 247us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.038211 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.038513 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":133,"threads_started":1}
I20260927 10:48:31.040598 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10561
I20260927 10:48:31.040712 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:31.040804 10692 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37509
I20260927 10:48:31.040958 10692 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:31.041054 9374 external_mini_cluster.cc:1428] Setting key 1944939a7aab00583cfb1ef20a264927
I20260927 10:48:31.041507 10692 heartbeater.cc:511] Master 127.9.39.190:37509 requested a full tablet report, sending...
I20260927 10:48:31.043112 10488 ts_manager.cc:194] Registered new tserver with Master: 549a6de2744f45af826324208435cffb (127.9.39.129:39953)
I20260927 10:48:31.044625 10488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:35819
I20260927 10:48:31.050567 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.043208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:45863 (local address 127.9.39.190:37509)
0927 10:48:31.043326 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.043330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.043442 (+ 112us) server_negotiation.cc:407] Connection header received
0927 10:48:31.043578 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.043581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.043625 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.043708 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.044116 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.044619 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.045891 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.046200 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.046314 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.047616 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.047619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.047621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.047640 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.048997 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.049497 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.049500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.049501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.049566 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.049869 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.049871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.049872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.050015 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.050086 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.050386 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.050464 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":53}
I20260927 10:48:31.050732 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.042857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.9.39.129:45863)
0927 10:48:31.043353 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.043369 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.043481 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.043722 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.043725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.043921 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.044005 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.044010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.044766 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.044769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.045758 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.045770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.046325 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.046331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.046487 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.047007 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.047027 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.047484 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.049110 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.049115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.049121 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.049407 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.049652 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.049655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.049657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.049757 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.050113 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.050119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.050217 (+ 98us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.050346 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.050491 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:31.070585 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37509
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:31.176678 10701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:31.176909 10701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:31.176961 10701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:31.180814 10701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:31.180862 10701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:31.180929 10701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:48:31.181011 10701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:31.185962 10701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:37509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:31.187048 10701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:31.187933 10701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:31.197042 10707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:31.197124 10706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:31.197057 10709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:31.197245 10701 server_base.cc:1034] running on GCE node
I20260927 10:48:31.197722 10701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:31.198350 10701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:31.199538 10701 hybrid_clock.cc:648] HybridClock initialized: now 1790506111199502 us; error 56 us; skew 500 ppm
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:31.202039 10701 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:31.202919 10701 webserver.cc:460] Webserver started at http://127.9.39.130:32859/ using document root <none> and password file <none>
I20260927 10:48:31.203441 10701 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:31.203536 10701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:31.203751 10701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:31.205726 10701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "0fe3f4446b5d41c0981accc3c5c84ebf"
format_stamp: "Formatted at 2026-09-27 10:48:31 on dist-test-slave-p9q6"
server_key: "a181fbec8a8634eff191ee8a404c055a"
server_key_iv: "c1e33228891e4886b740efd01130b888"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.206288 10701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "0fe3f4446b5d41c0981accc3c5c84ebf"
format_stamp: "Formatted at 2026-09-27 10:48:31 on dist-test-slave-p9q6"
server_key: "a181fbec8a8634eff191ee8a404c055a"
server_key_iv: "c1e33228891e4886b740efd01130b888"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.209807 10701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 10:48:31.212122 10716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:31.213248 10701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 10:48:31.213377 10701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "0fe3f4446b5d41c0981accc3c5c84ebf"
format_stamp: "Formatted at 2026-09-27 10:48:31 on dist-test-slave-p9q6"
server_key: "a181fbec8a8634eff191ee8a404c055a"
server_key_iv: "c1e33228891e4886b740efd01130b888"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.213478 10701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:31.233708 10701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:31.236572 10701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:31.236764 10701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:31.238198 10701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:31.238270 10701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:31.238333 10701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:31.238368 10701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:31.249199 10701 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:39623
I20260927 10:48:31.249301 10831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:39623 every 8 connection(s)
I20260927 10:48:31.250098 10701 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:31.252249 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.239123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:37379 (local address 127.9.39.190:37509)
0927 10:48:31.239256 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.239268 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.240303 (+ 1035us) server_negotiation.cc:407] Connection header received
0927 10:48:31.241638 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.241642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.241726 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.241821 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.243037 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.243649 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.244976 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.245308 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.245437 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.248483 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.248487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.248488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.248513 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.250102 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.250812 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.250819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.250822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.250919 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.251266 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.251268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.251269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.251449 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.251535 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.251984 (+ 449us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.252092 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:48:31.252893 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.239662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.9.39.130:37379)
0927 10:48:31.240126 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.240166 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.241407 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.242029 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.242035 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.242354 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.242883 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.242905 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.243803 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.243807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.244844 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.244855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.245495 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.245500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.245667 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.246362 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.246385 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.248319 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.250244 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.250249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.250262 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.250649 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.251002 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.251007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.251009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.251167 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.251550 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.251555 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.251759 (+ 204us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.251916 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.252145 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
I20260927 10:48:31.253949 10832 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37509
I20260927 10:48:31.254081 10832 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:31.254626 10832 heartbeater.cc:511] Master 127.9.39.190:37509 requested a full tablet report, sending...
I20260927 10:48:31.255730 10489 ts_manager.cc:194] Registered new tserver with Master: 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623)
I20260927 10:48:31.256342 10489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:37379
I20260927 10:48:31.256582 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10701
I20260927 10:48:31.256708 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:31.257025 9374 external_mini_cluster.cc:1428] Setting key 8babd1c6a0ac1ec5dbbbc4a06a662f70
I20260927 10:48:31.264892 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.256377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:32997 (local address 127.9.39.190:37509)
0927 10:48:31.257477 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.257483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.257499 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:48:31.257571 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.257574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.257620 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.257713 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.258160 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.258893 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.259714 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.260063 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.260147 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.261444 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.261446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.261448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.261468 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.262960 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.263525 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.263532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.263535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.263694 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.264097 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.264100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.264101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.264303 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.264382 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.264702 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.264782 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015}
I20260927 10:48:31.265313 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.256298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.9.39.130:32997)
0927 10:48:31.256655 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.256673 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.256782 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.257737 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.257740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.257932 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.258025 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.258031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.259030 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.259033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.259588 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.259596 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.260195 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.260202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.260323 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.260786 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.260808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.261314 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.263087 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.263091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.263095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.263348 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.263868 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.263873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.263878 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.263991 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.264408 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.264413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.264530 (+ 117us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.264664 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.264832 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:31.285542 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37509
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:31.392554 10841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:31.392781 10841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:31.392822 10841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:31.396755 10841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:31.396806 10841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:31.396842 10841 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:48:31.396873 10841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:31.401880 10841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:37509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:31.402741 10841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:31.403640 10841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:31.412797 10847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:31.412797 10849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:31.412793 10846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:31.412890 10841 server_base.cc:1034] running on GCE node
I20260927 10:48:31.413515 10841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:31.414096 10841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:31.415266 10841 hybrid_clock.cc:648] HybridClock initialized: now 1790506111415227 us; error 30 us; skew 500 ppm
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:31.417903 10841 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:31.418787 10841 webserver.cc:460] Webserver started at http://127.9.39.131:40331/ using document root <none> and password file <none>
I20260927 10:48:31.419304 10841 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:31.419394 10841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:31.419613 10841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:31.421464 10841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2"
format_stamp: "Formatted at 2026-09-27 10:48:31 on dist-test-slave-p9q6"
server_key: "d20a3fe3ef9e7adb96fdc3a76dee9892"
server_key_iv: "76ebac5734981d8129644b36cc107888"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.421980 10841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2"
format_stamp: "Formatted at 2026-09-27 10:48:31 on dist-test-slave-p9q6"
server_key: "d20a3fe3ef9e7adb96fdc3a76dee9892"
server_key_iv: "76ebac5734981d8129644b36cc107888"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.425470 10841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:48:31.427817 10856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:31.428848 10841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:31.428972 10841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2"
format_stamp: "Formatted at 2026-09-27 10:48:31 on dist-test-slave-p9q6"
server_key: "d20a3fe3ef9e7adb96fdc3a76dee9892"
server_key_iv: "76ebac5734981d8129644b36cc107888"
server_key_version: "encryptionkey@0"
I20260927 10:48:31.429077 10841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:31.439958 10841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:31.442943 10841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:31.443128 10841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:31.444491 10841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:31.444561 10841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:31.444628 10841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:31.444684 10841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:31.455456 10841 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:43143
I20260927 10:48:31.455523 10971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:43143 every 8 connection(s)
I20260927 10:48:31.456513 10841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:31.457902 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.445124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:41495 (local address 127.9.39.190:37509)
0927 10:48:31.445285 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.445292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.446061 (+ 769us) server_negotiation.cc:407] Connection header received
0927 10:48:31.447003 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.447007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.447065 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.447164 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.448463 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.449339 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.450687 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.451128 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.451245 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.454185 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.454188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.454191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.454222 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.455875 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.456328 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.456332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.456333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.456425 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.456795 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.456798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.456801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.457044 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.457146 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.457692 (+ 546us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.457791 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:48:31.458804 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.445472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.9.39.131:41495)
0927 10:48:31.445897 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.445944 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.446791 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.447456 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.447466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.447904 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.448342 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.448355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.449484 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.449487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.450583 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.450592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.451350 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.451355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.451543 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.452192 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.452218 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.454028 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.455975 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.455979 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.455990 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.456227 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.456521 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.456523 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.456524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.456702 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.457202 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.457211 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.457479 (+ 268us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.457631 (+ 152us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.457861 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":121,"threads_started":1}
I20260927 10:48:31.459933 10972 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37509
I20260927 10:48:31.460081 10972 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:31.460522 10972 heartbeater.cc:511] Master 127.9.39.190:37509 requested a full tablet report, sending...
I20260927 10:48:31.461192 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 10841
I20260927 10:48:31.461319 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:31.461553 10488 ts_manager.cc:194] Registered new tserver with Master: ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143)
I20260927 10:48:31.461585 9374 external_mini_cluster.cc:1428] Setting key f82015c9c5b450f1bcd7e98d47c4b2b8
I20260927 10:48:31.462304 10488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:41495
I20260927 10:48:31.463672 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:48:31.469769 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.462078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.9.39.131:40393)
0927 10:48:31.462434 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.462449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.462645 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.463010 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.463013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.463173 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.463260 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.463265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.464252 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.464256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.464832 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.464840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.465460 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.465465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.465581 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.466054 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.466070 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.466500 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.468247 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.468252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.468254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.468559 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.468829 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.468833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.468837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.468940 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.469368 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.469371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.469444 (+ 73us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.469528 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.469626 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":122,"threads_started":1}
I20260927 10:48:31.469990 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.462261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:40393 (local address 127.9.39.190:37509)
0927 10:48:31.462451 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.462457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.462508 (+ 51us) server_negotiation.cc:407] Connection header received
0927 10:48:31.462784 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.462789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.462834 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.462909 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.463376 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.464113 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.464944 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.465331 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.465449 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.466630 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.466633 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.466635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.466654 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.468138 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.468654 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.468657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.468658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.468722 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.469040 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.469045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.469047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.469268 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.469375 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.469614 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.469743 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:31.489612 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.481961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.9.39.190:37509)
0927 10:48:31.482078 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.482081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.482093 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:48:31.482125 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.482127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.482161 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.482235 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.482934 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.483460 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.484224 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.484454 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.484589 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.487038 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.487040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.487042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.487064 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.488221 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.488617 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.488619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.488621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.488692 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.488952 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.488955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.488956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.489112 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.489211 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.489415 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.489502 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:48:31.492355 10488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:31.495272 10488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:31.508821 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.502889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:39623 (local address 127.0.0.1:59640)
0927 10:48:31.503304 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.503346 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.503465 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.504468 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.504474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.504496 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.504640 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.504647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.505905 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.505909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.507334 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.507347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.508269 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.508272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.508399 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.508444 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.508555 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.508674 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":6,"thread_start_us":76,"threads_started":1}
I20260927 10:48:31.508852 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.503007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.9.39.130:39623)
0927 10:48:31.503938 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.503967 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.503995 (+ 28us) server_negotiation.cc:407] Connection header received
0927 10:48:31.504091 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.504096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.504281 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.504496 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.504768 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.505791 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.507473 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.508143 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.508268 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.508436 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.508591 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.508683 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":101,"threads_started":1}
I20260927 10:48:31.509871 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.503568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:39953 (local address 127.0.0.1:33098)
0927 10:48:31.504371 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.504385 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.504471 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.505302 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.505304 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.505321 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.505441 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.505450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.506890 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.506893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.508412 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.508422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.509478 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.509482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.509563 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.509588 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.509655 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.509736 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":252,"thread_start_us":77,"threads_started":1}
I20260927 10:48:31.510329 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.503989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.9.39.129:39953)
0927 10:48:31.504360 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.504371 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.504764 (+ 393us) server_negotiation.cc:407] Connection header received
0927 10:48:31.504834 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.504838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.505010 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.505158 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.505567 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.506463 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.508522 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.509378 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.509480 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.510030 (+ 550us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.510084 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.510171 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
I20260927 10:48:31.510162 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.502933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43143 (local address 127.0.0.1:54970)
0927 10:48:31.503672 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.503683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.503753 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.505153 (+ 1400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.505156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.505175 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.505281 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.505287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.506547 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.506550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.508741 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.508750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.509806 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.509810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.509911 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.509936 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.509986 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.510049 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":77,"threads_started":1}
I20260927 10:48:31.510813 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.503507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.9.39.131:43143)
0927 10:48:31.504706 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.504718 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.504741 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:48:31.504824 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.504829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.505011 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.505158 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.505399 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.506417 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.508877 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.509666 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.510258 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.510421 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.510469 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.510549 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":105,"threads_started":1}
I20260927 10:48:31.512099 10766 tablet_service.cc:1467] Processing CreateTablet for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a (DEFAULT_TABLE table=test-table [id=55cc396390754656b8f7aa7fae427fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:31.513236 10766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.513228 10906 tablet_service.cc:1467] Processing CreateTablet for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a (DEFAULT_TABLE table=test-table [id=55cc396390754656b8f7aa7fae427fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:31.514171 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.518981 10626 tablet_service.cc:1467] Processing CreateTablet for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a (DEFAULT_TABLE table=test-table [id=55cc396390754656b8f7aa7fae427fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:31.519754 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.521360 10996 tablet_bootstrap.cc:492] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Bootstrap starting.
I20260927 10:48:31.521736 10995 tablet_bootstrap.cc:492] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Bootstrap starting.
I20260927 10:48:31.523936 10996 tablet_bootstrap.cc:654] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.523947 10995 tablet_bootstrap.cc:654] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.524699 10996 log.cc:824] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:31.524830 10995 log.cc:824] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:31.526336 10999 tablet_bootstrap.cc:492] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Bootstrap starting.
I20260927 10:48:31.526571 10995 tablet_bootstrap.cc:492] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: No bootstrap required, opened a new log
I20260927 10:48:31.526643 10996 tablet_bootstrap.cc:492] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: No bootstrap required, opened a new log
I20260927 10:48:31.526772 10995 ts_tablet_manager.cc:1397] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:48:31.526844 10996 ts_tablet_manager.cc:1397] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260927 10:48:31.528527 10999 tablet_bootstrap.cc:654] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.529266 10999 log.cc:824] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:31.529765 10996 raft_consensus.cc:348] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.530058 10996 raft_consensus.cc:374] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.530113 10996 raft_consensus.cc:729] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe3f4446b5d41c0981accc3c5c84ebf, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.530421 10995 raft_consensus.cc:348] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.530547 10996 consensus_queue.cc:260] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.530643 10995 raft_consensus.cc:374] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.530690 10995 raft_consensus.cc:729] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae287c9f6ba1430f979ce6e3a9a1f9a2, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.531214 10999 tablet_bootstrap.cc:492] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: No bootstrap required, opened a new log
I20260927 10:48:31.531154 10995 consensus_queue.cc:260] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.531414 10999 ts_tablet_manager.cc:1397] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:48:31.531544 10996 ts_tablet_manager.cc:1428] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260927 10:48:31.531867 10972 heartbeater.cc:503] Master 127.9.39.190:37509 was elected leader, sending a full tablet report...
I20260927 10:48:31.531867 10832 heartbeater.cc:503] Master 127.9.39.190:37509 was elected leader, sending a full tablet report...
I20260927 10:48:31.533205 10995 ts_tablet_manager.cc:1428] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:31.535107 10999 raft_consensus.cc:348] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.535338 10999 raft_consensus.cc:374] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.535395 10999 raft_consensus.cc:729] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 549a6de2744f45af826324208435cffb, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.535810 10999 consensus_queue.cc:260] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.536424 10692 heartbeater.cc:503] Master 127.9.39.190:37509 was elected leader, sending a full tablet report...
I20260927 10:48:31.536697 10999 ts_tablet_manager.cc:1428] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
W20260927 10:48:31.537226 10693 tablet.cc:2369] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:31.694134 11001 raft_consensus.cc:482] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:31.694377 11001 raft_consensus.cc:504] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.695484 11001 leader_election.cc:288] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143), 549a6de2744f45af826324208435cffb (127.9.39.129:39953)
I20260927 10:48:31.699179 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.695929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:51091 (local address 127.9.39.131:43143)
0927 10:48:31.696081 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.696086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.696101 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:48:31.696274 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.696280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.696341 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.696422 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.696697 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.697211 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.698232 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.698623 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.698776 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.698830 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.698994 (+ 164us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.699063 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 10:48:31.699179 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.695842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43143 (local address 127.9.39.130:51091)
0927 10:48:31.695994 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.696011 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.696112 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.696433 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.696436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.696454 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.696584 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.696589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.697340 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.697343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.698107 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.698115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.698761 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.698765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.698863 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.698884 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.698980 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.699052 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":1}
I20260927 10:48:31.699885 11004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.696137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:39953 (local address 127.9.39.130:35701)
0927 10:48:31.696495 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.696506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.696616 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.696876 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.696879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.696894 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.697032 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.697037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.698047 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.698052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.698833 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.698842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.699587 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.699597 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.699663 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.699680 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.699740 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.699785 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":133,"threads_started":1}
I20260927 10:48:31.699838 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3dc985f2fa14d4e87fd3aaa42a8ac0a" candidate_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" is_pre_election: true
I20260927 10:48:31.699955 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.696284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:35701 (local address 127.9.39.129:39953)
0927 10:48:31.696432 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.696437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.696582 (+ 145us) server_negotiation.cc:407] Connection header received
0927 10:48:31.696729 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.696732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.696777 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.696865 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.697461 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.697938 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.698948 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.699454 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.699574 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.699618 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.699775 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.699848 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:48:31.700152 10926 raft_consensus.cc:2454] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fe3f4446b5d41c0981accc3c5c84ebf in term 0.
I20260927 10:48:31.700488 10646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3dc985f2fa14d4e87fd3aaa42a8ac0a" candidate_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "549a6de2744f45af826324208435cffb" is_pre_election: true
I20260927 10:48:31.700655 10717 leader_election.cc:302] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe3f4446b5d41c0981accc3c5c84ebf, ae287c9f6ba1430f979ce6e3a9a1f9a2; no voters:
I20260927 10:48:31.700775 10646 raft_consensus.cc:2454] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fe3f4446b5d41c0981accc3c5c84ebf in term 0.
I20260927 10:48:31.700942 11001 raft_consensus.cc:2781] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:31.701035 11001 raft_consensus.cc:482] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:31.701110 11001 raft_consensus.cc:3037] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:31.702102 11001 raft_consensus.cc:504] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
I20260927 10:48:31.702442 11001 leader_election.cc:288] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [CANDIDATE]: Term 1 election: Requested vote from peers ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143), 549a6de2744f45af826324208435cffb (127.9.39.129:39953)
I20260927 10:48:31.702819 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3dc985f2fa14d4e87fd3aaa42a8ac0a" candidate_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2"
I20260927 10:48:31.702840 10646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3dc985f2fa14d4e87fd3aaa42a8ac0a" candidate_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "549a6de2744f45af826324208435cffb"
I20260927 10:48:31.702939 10926 raft_consensus.cc:3037] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:31.702948 10646 raft_consensus.cc:3037] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:31.703922 10646 raft_consensus.cc:2454] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fe3f4446b5d41c0981accc3c5c84ebf in term 1.
I20260927 10:48:31.704195 10717 leader_election.cc:302] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe3f4446b5d41c0981accc3c5c84ebf, 549a6de2744f45af826324208435cffb; no voters:
I20260927 10:48:31.704319 10926 raft_consensus.cc:2454] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fe3f4446b5d41c0981accc3c5c84ebf in term 1.
I20260927 10:48:31.704344 11001 raft_consensus.cc:2781] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:31.704571 11001 raft_consensus.cc:686] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 LEADER]: Becoming Leader. State: Replica: 0fe3f4446b5d41c0981accc3c5c84ebf, State: Running, Role: LEADER
I20260927 10:48:31.704895 11001 consensus_queue.cc:237] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } }
W20260927 10:48:31.707444 10973 tablet.cc:2369] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:31.708086 10489 catalog_manager.cc:5495] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130). New cstate: current_term: 1 leader_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:31.744884 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.741637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.9.39.130:39623)
0927 10:48:31.741761 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.741765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.741795 (+ 30us) server_negotiation.cc:407] Connection header received
0927 10:48:31.741992 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.741994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.742065 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.742144 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:31.742381 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.742911 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.743690 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.743881 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.743948 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.744200 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:31.744569 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:31.744648 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.744739 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.744784 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:48:31.750283 10926 raft_consensus.cc:1261] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Refusing update from remote peer 0fe3f4446b5d41c0981accc3c5c84ebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:31.750277 10646 raft_consensus.cc:1261] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Refusing update from remote peer 0fe3f4446b5d41c0981accc3c5c84ebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:31.750964 11001 consensus_queue.cc:1035] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:31.751145 11005 consensus_queue.cc:1035] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 10:48:31.751530 10833 tablet.cc:2369] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:31.758759 11010 mvcc.cc:205] Tried to move back new op lower bound from 7333913033722191872 to 7333913033545560064. Current Snapshot: MvccSnapshot[applied={T|T < 7333913033722191872}]
I20260927 10:48:31.769169 10489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:31.769325 10489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:31.771406 10489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: Sending DeleteTablet for 3 replicas of tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a
I20260927 10:48:31.772056 10766 tablet_service.cc:1514] Processing DeleteTablet for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:31 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:59640
I20260927 10:48:31.772600 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:31 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:33098
I20260927 10:48:31.772610 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:31 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:54970
I20260927 10:48:31.773205 10489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:31.773566 10489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:31.774183 11019 tablet_replica.cc:330] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: stopping tablet replica
I20260927 10:48:31.774457 11019 raft_consensus.cc:2229] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:31.774744 11019 raft_consensus.cc:2258] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:31.775060 11018 tablet_replica.cc:330] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: stopping tablet replica
I20260927 10:48:31.775723 11018 raft_consensus.cc:2229] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:31.776146 11018 raft_consensus.cc:2258] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:31.777644 11018 ts_tablet_manager.cc:1905] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:31.778255 10766 tablet_service.cc:1467] Processing CreateTablet for tablet 49ebed3971c14668a5e5ebd2dfc34184 (DEFAULT_TABLE table=test-table [id=09954df3b4404a8fae00fddc5f78f7be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:31.778563 10766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ebed3971c14668a5e5ebd2dfc34184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.779063 11019 ts_tablet_manager.cc:1905] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:31.781064 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 49ebed3971c14668a5e5ebd2dfc34184 (DEFAULT_TABLE table=test-table [id=09954df3b4404a8fae00fddc5f78f7be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:31.781280 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ebed3971c14668a5e5ebd2dfc34184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.781587 11019 ts_tablet_manager.cc:1918] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:31.781687 11019 log.cc:1196] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/d3dc985f2fa14d4e87fd3aaa42a8ac0a
I20260927 10:48:31.781956 11019 ts_tablet_manager.cc:1939] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P ae287c9f6ba1430f979ce6e3a9a1f9a2: Deleting consensus metadata
I20260927 10:48:31.782991 10473 catalog_manager.cc:4841] TS ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143): tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a (table test-table [id=55cc396390754656b8f7aa7fae427fa5]) successfully deleted
I20260927 10:48:31.783335 10995 tablet_bootstrap.cc:492] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Bootstrap starting.
I20260927 10:48:31.783402 10996 tablet_bootstrap.cc:492] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Bootstrap starting.
I20260927 10:48:31.784278 10996 tablet_bootstrap.cc:654] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.784245 10995 tablet_bootstrap.cc:654] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.785185 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 49ebed3971c14668a5e5ebd2dfc34184 (DEFAULT_TABLE table=test-table [id=09954df3b4404a8fae00fddc5f78f7be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:31.785333 10995 tablet_bootstrap.cc:492] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: No bootstrap required, opened a new log
I20260927 10:48:31.785409 10995 ts_tablet_manager.cc:1397] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:31.785418 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ebed3971c14668a5e5ebd2dfc34184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.785835 10995 raft_consensus.cc:348] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.785955 10995 raft_consensus.cc:374] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.785995 10995 raft_consensus.cc:729] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae287c9f6ba1430f979ce6e3a9a1f9a2, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.786127 10995 consensus_queue.cc:260] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.786413 10995 ts_tablet_manager.cc:1428] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:31.787647 10999 tablet_bootstrap.cc:492] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Bootstrap starting.
I20260927 10:48:31.788543 10999 tablet_bootstrap.cc:654] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.788748 10996 tablet_bootstrap.cc:492] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: No bootstrap required, opened a new log
I20260927 10:48:31.788815 10996 ts_tablet_manager.cc:1397] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260927 10:48:31.789215 10996 raft_consensus.cc:348] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.789317 10996 raft_consensus.cc:374] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.789356 10996 raft_consensus.cc:729] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe3f4446b5d41c0981accc3c5c84ebf, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.789460 10996 consensus_queue.cc:260] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.789685 10996 ts_tablet_manager.cc:1428] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:31.790337 11017 tablet_replica.cc:330] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: stopping tablet replica
I20260927 10:48:31.790616 11017 raft_consensus.cc:2229] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:31.790956 11017 raft_consensus.cc:2258] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:31.791677 11018 ts_tablet_manager.cc:1918] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:31.791793 11018 log.cc:1196] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/d3dc985f2fa14d4e87fd3aaa42a8ac0a
I20260927 10:48:31.792057 11018 ts_tablet_manager.cc:1939] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 549a6de2744f45af826324208435cffb: Deleting consensus metadata
I20260927 10:48:31.792167 11017 ts_tablet_manager.cc:1905] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:31.793027 10473 catalog_manager.cc:4841] TS 549a6de2744f45af826324208435cffb (127.9.39.129:39953): tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a (table test-table [id=55cc396390754656b8f7aa7fae427fa5]) successfully deleted
I20260927 10:48:31.793496 10999 tablet_bootstrap.cc:492] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: No bootstrap required, opened a new log
I20260927 10:48:31.793613 10999 ts_tablet_manager.cc:1397] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260927 10:48:31.794106 10999 raft_consensus.cc:348] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.794209 10999 raft_consensus.cc:374] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.794245 10999 raft_consensus.cc:729] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 549a6de2744f45af826324208435cffb, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.794368 10999 consensus_queue.cc:260] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.794497 11017 ts_tablet_manager.cc:1918] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:31.794577 11017 log.cc:1196] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/d3dc985f2fa14d4e87fd3aaa42a8ac0a
I20260927 10:48:31.794682 10999 ts_tablet_manager.cc:1428] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260927 10:48:31.794880 11017 ts_tablet_manager.cc:1939] T d3dc985f2fa14d4e87fd3aaa42a8ac0a P 0fe3f4446b5d41c0981accc3c5c84ebf: Deleting consensus metadata
I20260927 10:48:31.795784 10475 catalog_manager.cc:4841] TS 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623): tablet d3dc985f2fa14d4e87fd3aaa42a8ac0a (table test-table [id=55cc396390754656b8f7aa7fae427fa5]) successfully deleted
I20260927 10:48:31.843921 11003 raft_consensus.cc:482] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:31.844062 11003 raft_consensus.cc:504] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.845160 11003 leader_election.cc:288] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623), ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143)
I20260927 10:48:31.849463 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.845424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43143 (local address 127.9.39.129:43415)
0927 10:48:31.845844 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.845860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.845978 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.846341 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.846344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.846362 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.846483 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.846488 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.847497 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.847501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.848344 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.848352 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.849029 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.849032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.849138 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.849163 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.849238 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.849313 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":84,"threads_started":1}
I20260927 10:48:31.849471 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.845618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:43415 (local address 127.9.39.131:43143)
0927 10:48:31.845777 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.845782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.845927 (+ 145us) server_negotiation.cc:407] Connection header received
0927 10:48:31.846105 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.846108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.846156 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.846243 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.846599 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.847355 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.848471 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.848894 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.849008 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.849060 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.849276 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.849347 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:48:31.850093 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.845343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:39623 (local address 127.9.39.129:50639)
0927 10:48:31.845817 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.845831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.845961 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.846237 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.846240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.846256 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:31.846391 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.846397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.847462 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.847466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.848758 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.848768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.849647 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.849651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.849749 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.849765 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.849870 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.849948 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":33,"thread_start_us":89,"threads_started":1}
I20260927 10:48:31.850095 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.845532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:50639 (local address 127.9.39.130:39623)
0927 10:48:31.845685 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.845690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.845927 (+ 237us) server_negotiation.cc:407] Connection header received
0927 10:48:31.846073 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.846076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.846122 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.846206 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:31.846511 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.847341 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.848892 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.849421 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.849536 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.849584 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.849913 (+ 329us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.849986 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:48:31.850195 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ebed3971c14668a5e5ebd2dfc34184" candidate_uuid: "549a6de2744f45af826324208435cffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" is_pre_election: true
I20260927 10:48:31.850343 10926 raft_consensus.cc:2454] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 549a6de2744f45af826324208435cffb in term 0.
I20260927 10:48:31.850752 10577 leader_election.cc:302] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 549a6de2744f45af826324208435cffb, ae287c9f6ba1430f979ce6e3a9a1f9a2; no voters:
I20260927 10:48:31.850912 10786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ebed3971c14668a5e5ebd2dfc34184" candidate_uuid: "549a6de2744f45af826324208435cffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" is_pre_election: true
I20260927 10:48:31.851001 11003 raft_consensus.cc:2781] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:31.851052 11003 raft_consensus.cc:482] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:31.851094 11003 raft_consensus.cc:3037] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:31.851138 10786 raft_consensus.cc:2454] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 549a6de2744f45af826324208435cffb in term 0.
I20260927 10:48:31.852195 11003 raft_consensus.cc:504] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.852591 11003 leader_election.cc:288] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [CANDIDATE]: Term 1 election: Requested vote from peers 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623), ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143)
I20260927 10:48:31.852864 10786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ebed3971c14668a5e5ebd2dfc34184" candidate_uuid: "549a6de2744f45af826324208435cffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf"
I20260927 10:48:31.852991 10786 raft_consensus.cc:3037] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:31.852999 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ebed3971c14668a5e5ebd2dfc34184" candidate_uuid: "549a6de2744f45af826324208435cffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2"
I20260927 10:48:31.853127 10926 raft_consensus.cc:3037] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:31.854146 10786 raft_consensus.cc:2454] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 549a6de2744f45af826324208435cffb in term 1.
I20260927 10:48:31.854216 10926 raft_consensus.cc:2454] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 549a6de2744f45af826324208435cffb in term 1.
I20260927 10:48:31.854501 10577 leader_election.cc:302] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 549a6de2744f45af826324208435cffb, ae287c9f6ba1430f979ce6e3a9a1f9a2; no voters:
I20260927 10:48:31.854771 11003 raft_consensus.cc:2781] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:31.854985 11003 raft_consensus.cc:686] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 1 LEADER]: Becoming Leader. State: Replica: 549a6de2744f45af826324208435cffb, State: Running, Role: LEADER
I20260927 10:48:31.855221 11003 consensus_queue.cc:237] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } }
I20260927 10:48:31.857324 10489 catalog_manager.cc:5495] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb reported cstate change: term changed from 0 to 1, leader changed from <none> to 549a6de2744f45af826324208435cffb (127.9.39.129). New cstate: current_term: 1 leader_uuid: "549a6de2744f45af826324208435cffb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } health_report { overall_health: UNKNOWN } } }
Sep 27 10:48:31 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:31.901052 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.892318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37509 (local address 127.0.0.1:38916)
0927 10:48:31.892588 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:31.892604 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:31.892707 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:31.892983 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:31.892985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:31.893356 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:31.893442 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.893448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.894518 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.894521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.895103 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:31.895111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.895742 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.895745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:31.895809 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.896439 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:31.896451 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:31.897767 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:31.899628 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.899636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.899640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.899847 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.900140 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:31.900142 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:31.900143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:31.900213 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:31.900646 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:31.900651 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:31.900738 (+ 87us) client_negotiation.cc:769] Sending connection context
0927 10:48:31.900820 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 10:48:31.900925 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
I20260927 10:48:31.901204 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:31.892423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.9.39.190:37509)
0927 10:48:31.892558 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:31.892563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:31.892702 (+ 139us) server_negotiation.cc:407] Connection header received
0927 10:48:31.892811 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:31.892814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:31.892866 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:31.892948 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:31.893578 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.894367 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.895275 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:31.895615 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:31.895738 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:31.897907 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:31.897912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:31.897914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:31.897938 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:31.899527 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.899950 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.899954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.899956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.900046 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:31.900316 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:31.900319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:31.900321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:31.900537 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:31.900640 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:31.900918 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 10:48:31.901030 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:48:31.904883 10489 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:38916:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 10:48:31.905571 10489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:31.910295 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 9e1d7bda008c4f18b0995dc3e3bb34dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11cc299b2e784e698681378f8f710853]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:48:31.910462 10766 tablet_service.cc:1467] Processing CreateTablet for tablet 9e1d7bda008c4f18b0995dc3e3bb34dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11cc299b2e784e698681378f8f710853]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:48:31.910462 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 9e1d7bda008c4f18b0995dc3e3bb34dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11cc299b2e784e698681378f8f710853]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:48:31.910670 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e1d7bda008c4f18b0995dc3e3bb34dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.910832 10766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e1d7bda008c4f18b0995dc3e3bb34dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.910832 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e1d7bda008c4f18b0995dc3e3bb34dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:31.913479 10996 tablet_bootstrap.cc:492] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: Bootstrap starting.
I20260927 10:48:31.913537 10999 tablet_bootstrap.cc:492] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: Bootstrap starting.
I20260927 10:48:31.914513 10999 tablet_bootstrap.cc:654] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.914939 10996 tablet_bootstrap.cc:654] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.915381 10995 tablet_bootstrap.cc:492] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: Bootstrap starting.
I20260927 10:48:31.916472 10995 tablet_bootstrap.cc:654] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:31.916471 10996 tablet_bootstrap.cc:492] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: No bootstrap required, opened a new log
I20260927 10:48:31.916579 10996 ts_tablet_manager.cc:1397] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:31.917085 10996 raft_consensus.cc:348] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:31.917222 10996 raft_consensus.cc:374] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.917268 10996 raft_consensus.cc:729] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe3f4446b5d41c0981accc3c5c84ebf, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.917395 10996 consensus_queue.cc:260] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:31.917706 10996 ts_tablet_manager.cc:1428] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 10:48:31.917851 10995 tablet_bootstrap.cc:492] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: No bootstrap required, opened a new log
I20260927 10:48:31.917918 10995 ts_tablet_manager.cc:1397] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:31.917661 11001 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:31.918236 11001 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:31.918336 10995 raft_consensus.cc:348] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:31.918462 10995 raft_consensus.cc:374] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.918503 10995 raft_consensus.cc:729] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae287c9f6ba1430f979ce6e3a9a1f9a2, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.918612 10995 consensus_queue.cc:260] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:31.918963 10995 ts_tablet_manager.cc:1428] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 10:48:31.919165 11002 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:31.919324 11002 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:31.920377 10999 tablet_bootstrap.cc:492] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: No bootstrap required, opened a new log
I20260927 10:48:31.920470 10999 ts_tablet_manager.cc:1397] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260927 10:48:31.920944 10999 raft_consensus.cc:348] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:31.921077 10999 raft_consensus.cc:374] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:31.921125 10999 raft_consensus.cc:729] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 549a6de2744f45af826324208435cffb, State: Initialized, Role: FOLLOWER
I20260927 10:48:31.921235 10999 consensus_queue.cc:260] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:31.921504 10999 ts_tablet_manager.cc:1428] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 10:48:31.921449 11003 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:31.921584 11003 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:32.103627 11002 raft_consensus.cc:482] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:32.103778 11002 raft_consensus.cc:504] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:32.105013 11002 leader_election.cc:288] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 549a6de2744f45af826324208435cffb (127.9.39.129:39953), 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623)
I20260927 10:48:32.110509 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.105372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:33623 (local address 127.9.39.129:39953)
0927 10:48:32.105510 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.105515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.105695 (+ 180us) server_negotiation.cc:407] Connection header received
0927 10:48:32.105886 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.105889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.105933 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.106019 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:32.106461 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.107306 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.109137 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.109798 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.109902 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.109943 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.110312 (+ 369us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.110386 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 10:48:32.110509 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.105386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34669 (local address 127.9.39.130:39623)
0927 10:48:32.105618 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.105622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.105695 (+ 73us) server_negotiation.cc:407] Connection header received
0927 10:48:32.105876 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.105879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.105933 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.106020 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:32.106461 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.107286 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.109136 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.109767 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.109894 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.109943 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.110312 (+ 369us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.110386 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260927 10:48:32.110617 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.105262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:39623 (local address 127.9.39.131:34669)
0927 10:48:32.105580 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:32.105598 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:32.105746 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:32.106076 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:32.106080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:32.106108 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:32.106322 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.106332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.107413 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.107417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.108987 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.109000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.109910 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.109914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.110150 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.110204 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 10:48:32.110295 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 10:48:32.110387 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":301,"spinlock_wait_cycles":33920,"thread_start_us":93,"threads_started":1}
I20260927 10:48:32.110870 11036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.105263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:39953 (local address 127.9.39.131:33623)
0927 10:48:32.105580 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:32.105598 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:32.105746 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:32.106076 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:32.106080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:32.106108 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:32.106331 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.106336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.107413 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.107417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.108997 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.109002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.109910 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.109914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.110144 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.110202 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 10:48:32.110281 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 10:48:32.110387 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":392,"spinlock_wait_cycles":19328,"thread_start_us":73,"threads_started":1}
I20260927 10:48:32.111130 10786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e1d7bda008c4f18b0995dc3e3bb34dd" candidate_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" is_pre_election: true
I20260927 10:48:32.111310 10786 raft_consensus.cc:2454] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae287c9f6ba1430f979ce6e3a9a1f9a2 in term 0.
I20260927 10:48:32.111310 10646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e1d7bda008c4f18b0995dc3e3bb34dd" candidate_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "549a6de2744f45af826324208435cffb" is_pre_election: true
I20260927 10:48:32.111424 10646 raft_consensus.cc:2454] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae287c9f6ba1430f979ce6e3a9a1f9a2 in term 0.
I20260927 10:48:32.111671 10859 leader_election.cc:302] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe3f4446b5d41c0981accc3c5c84ebf, ae287c9f6ba1430f979ce6e3a9a1f9a2; no voters:
I20260927 10:48:32.111932 11002 raft_consensus.cc:2781] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:32.112006 11002 raft_consensus.cc:482] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:32.112061 11002 raft_consensus.cc:3037] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:32.113044 11002 raft_consensus.cc:504] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:32.113372 11002 leader_election.cc:288] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [CANDIDATE]: Term 1 election: Requested vote from peers 549a6de2744f45af826324208435cffb (127.9.39.129:39953), 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623)
I20260927 10:48:32.113689 10786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e1d7bda008c4f18b0995dc3e3bb34dd" candidate_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf"
I20260927 10:48:32.113689 10646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e1d7bda008c4f18b0995dc3e3bb34dd" candidate_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "549a6de2744f45af826324208435cffb"
I20260927 10:48:32.113808 10786 raft_consensus.cc:3037] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:32.113822 10646 raft_consensus.cc:3037] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:32.114861 10646 raft_consensus.cc:2454] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae287c9f6ba1430f979ce6e3a9a1f9a2 in term 1.
I20260927 10:48:32.114861 10786 raft_consensus.cc:2454] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae287c9f6ba1430f979ce6e3a9a1f9a2 in term 1.
I20260927 10:48:32.115125 10859 leader_election.cc:302] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe3f4446b5d41c0981accc3c5c84ebf, ae287c9f6ba1430f979ce6e3a9a1f9a2; no voters:
I20260927 10:48:32.115300 11002 raft_consensus.cc:2781] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:32.115496 11002 raft_consensus.cc:686] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 LEADER]: Becoming Leader. State: Replica: ae287c9f6ba1430f979ce6e3a9a1f9a2, State: Running, Role: LEADER
I20260927 10:48:32.115742 11002 consensus_queue.cc:237] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } }
I20260927 10:48:32.117420 11038 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: TxnStatusTablet state changed. Reason: New leader ae287c9f6ba1430f979ce6e3a9a1f9a2. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.117527 11038 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:48:32.117800 11040 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 10:48:32.117942 11040 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 10:48:32.118872 10488 catalog_manager.cc:5495] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131). New cstate: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:32.154063 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.149951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43143 (local address 127.0.0.1:54978)
0927 10:48:32.150092 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:32.150105 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:32.150214 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:32.150541 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:32.150544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:32.150555 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:32.150659 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.150665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.151450 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.151453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.152447 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.152457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.152888 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.152892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.152980 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.153030 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.153706 (+ 676us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.153727 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 10:48:32.153802 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 10:48:32.153892 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 10:48:32.153980 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.150060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.9.39.131:43143)
0927 10:48:32.150190 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.150193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.150206 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:48:32.150372 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.150374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.150425 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.150500 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:32.150804 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.151308 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.152564 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.152786 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.152861 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.153187 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.153611 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.153697 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.153823 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.153874 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 10:48:32.156929 10646 raft_consensus.cc:1261] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Refusing update from remote peer ae287c9f6ba1430f979ce6e3a9a1f9a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:32.157202 10786 raft_consensus.cc:1261] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Refusing update from remote peer ae287c9f6ba1430f979ce6e3a9a1f9a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:32.157557 11002 consensus_queue.cc:1035] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:32.157717 11038 consensus_queue.cc:1035] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:32.159423 11001 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: TxnStatusTablet state changed. Reason: New leader ae287c9f6ba1430f979ce6e3a9a1f9a2. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.159528 11001 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:32.160323 11003 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: TxnStatusTablet state changed. Reason: New leader ae287c9f6ba1430f979ce6e3a9a1f9a2. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.160446 11003 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:32.162305 11001 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.162403 11001 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 0fe3f4446b5d41c0981accc3c5c84ebf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:32.162429 11003 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.162479 11008 mvcc.cc:205] Tried to move back new op lower bound from 7333913035390013440 to 7333913035228712960. Current Snapshot: MvccSnapshot[applied={T|T < 7333913035390013440}]
I20260927 10:48:32.162520 11003 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P 549a6de2744f45af826324208435cffb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:32.164949 11038 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.165103 11038 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:48:32.164963 11002 tablet_replica.cc:439] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "549a6de2744f45af826324208435cffb" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39953 } } peers { permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 } } peers { permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 } } }
I20260927 10:48:32.165506 11002 tablet_replica.cc:442] T 9e1d7bda008c4f18b0995dc3e3bb34dd P ae287c9f6ba1430f979ce6e3a9a1f9a2: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:48:32.171514 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.167844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.9.39.129:39953)
0927 10:48:32.167942 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.167946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.167968 (+ 22us) server_negotiation.cc:407] Connection header received
0927 10:48:32.168032 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.168034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.168080 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.168165 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:32.168393 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.169168 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.169940 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.170283 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.170386 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.170607 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.171164 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.171279 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.171334 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.171386 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 10:48:32.181907 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.178220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:51905 (local address 127.9.39.131:43143)
0927 10:48:32.178368 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.178371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.178480 (+ 109us) server_negotiation.cc:407] Connection header received
0927 10:48:32.178598 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.178600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.178652 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.178825 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:32.179085 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.179703 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.180932 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.181153 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.181226 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.181526 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.181586 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.181649 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.181771 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.181810 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 10:48:32.181974 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.178102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43143 (local address 127.9.39.129:51905)
0927 10:48:32.178391 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:32.178406 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:32.178506 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:32.178830 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:32.178833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:32.178841 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:32.178985 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.178993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.179803 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.179806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.180805 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.180815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.181254 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.181258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.181353 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.181403 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.181673 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.181678 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:48:32.181749 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 10:48:32.181822 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260927 10:48:32.189472 10926 raft_consensus.cc:1261] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Refusing update from remote peer 549a6de2744f45af826324208435cffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:32.189487 10786 raft_consensus.cc:1261] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Refusing update from remote peer 549a6de2744f45af826324208435cffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:32.189962 11025 consensus_queue.cc:1035] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae287c9f6ba1430f979ce6e3a9a1f9a2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:32.190197 11003 consensus_queue.cc:1035] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fe3f4446b5d41c0981accc3c5c84ebf" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:32.196762 11009 mvcc.cc:205] Tried to move back new op lower bound from 7333913035523731456 to 7333913034161377280. Current Snapshot: MvccSnapshot[applied={T|T < 7333913035523731456}]
I20260927 10:48:32.212394 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.208420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:49747 (local address 127.9.39.129:39953)
0927 10:48:32.208610 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.208615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.208889 (+ 274us) server_negotiation.cc:407] Connection header received
0927 10:48:32.209020 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.209023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.209076 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.209138 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:32.209446 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.210158 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.211443 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.211648 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.211726 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.212032 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.212093 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.212145 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.212256 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.212296 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260927 10:48:32.212467 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.208340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:39953 (local address 127.9.39.131:49747)
0927 10:48:32.208793 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:32.208807 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:32.208922 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:32.209170 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:32.209173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:32.209180 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:32.209310 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.209316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.210254 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.210257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.211295 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:32.211306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.211756 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.211760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:32.211860 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.211911 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:32.212171 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:32.212177 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 10:48:32.212249 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 10:48:32.212325 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":94,"threads_started":1}
I20260927 10:48:32.234051 10488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:32.234207 10488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:32.236096 10488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d1cb625d7cd4e87ace7fe307c1b7d37: Sending DeleteTablet for 3 replicas of tablet 49ebed3971c14668a5e5ebd2dfc34184
I20260927 10:48:32.236599 10766 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ebed3971c14668a5e5ebd2dfc34184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:32 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:59640
I20260927 10:48:32.236618 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ebed3971c14668a5e5ebd2dfc34184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:32 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:33098
I20260927 10:48:32.236804 11017 tablet_replica.cc:330] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: stopping tablet replica
I20260927 10:48:32.236830 11018 tablet_replica.cc:330] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: stopping tablet replica
I20260927 10:48:32.236933 11017 raft_consensus.cc:2229] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:32.236932 11018 raft_consensus.cc:2229] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:32.237051 11017 raft_consensus.cc:2258] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:32.237223 11018 raft_consensus.cc:2258] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:32.237247 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ebed3971c14668a5e5ebd2dfc34184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:32 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:54970
I20260927 10:48:32.237839 11017 ts_tablet_manager.cc:1905] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:32.238152 11019 tablet_replica.cc:330] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: stopping tablet replica
I20260927 10:48:32.238274 11019 raft_consensus.cc:2229] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:32.238397 11019 raft_consensus.cc:2258] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:32.239391 11019 ts_tablet_manager.cc:1905] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:32.240789 11017 ts_tablet_manager.cc:1918] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:32.240862 11017 log.cc:1196] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/49ebed3971c14668a5e5ebd2dfc34184
I20260927 10:48:32.241143 11017 ts_tablet_manager.cc:1939] T 49ebed3971c14668a5e5ebd2dfc34184 P 0fe3f4446b5d41c0981accc3c5c84ebf: Deleting consensus metadata
I20260927 10:48:32.241895 10475 catalog_manager.cc:4841] TS 0fe3f4446b5d41c0981accc3c5c84ebf (127.9.39.130:39623): tablet 49ebed3971c14668a5e5ebd2dfc34184 (table test-table [id=09954df3b4404a8fae00fddc5f78f7be]) successfully deleted
I20260927 10:48:32.242080 11019 ts_tablet_manager.cc:1918] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:32.242139 11019 log.cc:1196] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/49ebed3971c14668a5e5ebd2dfc34184
I20260927 10:48:32.242388 11019 ts_tablet_manager.cc:1939] T 49ebed3971c14668a5e5ebd2dfc34184 P ae287c9f6ba1430f979ce6e3a9a1f9a2: Deleting consensus metadata
I20260927 10:48:32.242990 10473 catalog_manager.cc:4841] TS ae287c9f6ba1430f979ce6e3a9a1f9a2 (127.9.39.131:43143): tablet 49ebed3971c14668a5e5ebd2dfc34184 (table test-table [id=09954df3b4404a8fae00fddc5f78f7be]) successfully deleted
I20260927 10:48:32.244201 11018 ts_tablet_manager.cc:1905] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 27 10:48:32 dist-test-slave-p9q6 krb5kdc[10444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506111, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:32.247597 11018 ts_tablet_manager.cc:1918] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:32.248493 11018 log.cc:1196] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/49ebed3971c14668a5e5ebd2dfc34184
I20260927 10:48:32.248816 11018 ts_tablet_manager.cc:1939] T 49ebed3971c14668a5e5ebd2dfc34184 P 549a6de2744f45af826324208435cffb: Deleting consensus metadata
I20260927 10:48:32.249667 10473 catalog_manager.cc:4841] TS 549a6de2744f45af826324208435cffb (127.9.39.129:39953): tablet 49ebed3971c14668a5e5ebd2dfc34184 (table test-table [id=09954df3b4404a8fae00fddc5f78f7be]) successfully deleted
I20260927 10:48:32.250836 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.238252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.9.39.129:39953)
0927 10:48:32.238352 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.238357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.239016 (+ 659us) server_negotiation.cc:407] Connection header received
0927 10:48:32.239268 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.239271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.239316 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.239366 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:32.240347 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.241061 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.242668 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.243007 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.243079 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.246810 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:32.246823 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:32.246842 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:32.246867 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:32.248955 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:32.249425 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:32.249430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:32.249432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:32.249512 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:32.249837 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:32.249842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:32.249844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:32.250090 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:32.250215 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.250524 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.250635 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260927 10:48:32.251277 10686 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33108
I20260927 10:48:32.261421 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:32.252765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.9.39.190:37509)
0927 10:48:32.252895 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:32.252900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:32.253258 (+ 358us) server_negotiation.cc:407] Connection header received
0927 10:48:32.253375 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:32.253379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:32.253427 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:32.253520 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:32.254123 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.254796 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.255864 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:32.256195 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:32.256277 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:32.257861 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:32.257865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:32.257868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:32.257892 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:32.259687 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:32.260138 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:32.260142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:32.260143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:32.260234 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:32.260537 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:32.260540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:32.260542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:32.260723 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:32.260817 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:32.261154 (+ 337us) server_negotiation.cc:299] Negotiation successful
0927 10:48:32.261274 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260927 10:48:32.262097 10488 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38926
I20260927 10:48:32.263420 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10561
I20260927 10:48:32.270862 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10701
I20260927 10:48:32.278401 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10841
I20260927 10:48:32.284929 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 10457
2026-09-27T10:48:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2850 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:32 dist-test-slave-p9q6 krb5kdc[11061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:32 dist-test-slave-p9q6 krb5kdc[11061](info): set up 2 sockets
Sep 27 10:48:32 dist-test-slave-p9q6 krb5kdc[11061](info): commencing operation
krb5kdc: starting...
W20260927 10:48:33.314332 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:33 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:33.429783 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42861
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:33.532634 11074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:33.532914 11074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:33.532991 11074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:33.536834 11074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:33.536914 11074 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 10:48:33.536947 11074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:33.536988 11074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:33.537040 11074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:33.537075 11074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:33.542667 11074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:42861
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:33.543793 11074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:33.544762 11074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:33.552999 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:33.553023 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:33.553009 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:33.553087 11074 server_base.cc:1034] running on GCE node
I20260927 10:48:33.553828 11074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:33.554965 11074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:33.556171 11074 hybrid_clock.cc:648] HybridClock initialized: now 1790506113556157 us; error 28 us; skew 500 ppm
Sep 27 10:48:33 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506113, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:33.558840 11074 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:33.559711 11074 webserver.cc:460] Webserver started at http://127.9.39.190:41265/ using document root <none> and password file <none>
I20260927 10:48:33.560242 11074 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:33.560328 11074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:33.560541 11074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:33.562469 11074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "6a7e85e21c994ef68e4496f56d12b4c9"
format_stamp: "Formatted at 2026-09-27 10:48:33 on dist-test-slave-p9q6"
server_key: "8c11b32c6d6622cbf0bc57b20c25a6de"
server_key_iv: "602a4ce622c4bddfdc982b4fc0670f07"
server_key_version: "encryptionkey@0"
I20260927 10:48:33.563061 11074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "6a7e85e21c994ef68e4496f56d12b4c9"
format_stamp: "Formatted at 2026-09-27 10:48:33 on dist-test-slave-p9q6"
server_key: "8c11b32c6d6622cbf0bc57b20c25a6de"
server_key_iv: "602a4ce622c4bddfdc982b4fc0670f07"
server_key_version: "encryptionkey@0"
I20260927 10:48:33.566448 11074 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:33.568831 11089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:33.570037 11074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:33.570171 11074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "6a7e85e21c994ef68e4496f56d12b4c9"
format_stamp: "Formatted at 2026-09-27 10:48:33 on dist-test-slave-p9q6"
server_key: "8c11b32c6d6622cbf0bc57b20c25a6de"
server_key_iv: "602a4ce622c4bddfdc982b4fc0670f07"
server_key_version: "encryptionkey@0"
I20260927 10:48:33.570276 11074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:33.584229 11074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:33.587574 11074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:33.587724 11074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:33.595096 11074 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:42861
I20260927 10:48:33.595115 11151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:42861 every 8 connection(s)
I20260927 10:48:33.596005 11074 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:33.598526 11152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:33.603827 11152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Bootstrap starting.
I20260927 10:48:33.605104 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11074
I20260927 10:48:33.605259 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:33.605511 9374 external_mini_cluster.cc:1428] Setting key a63b9906474c08e1da967d98260f8cf4
I20260927 10:48:33.606482 11152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:33.607308 11152 log.cc:824] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:33.609076 11152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: No bootstrap required, opened a new log
I20260927 10:48:33.612311 11152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42861 } }
I20260927 10:48:33.612566 11152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:33.612654 11152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a7e85e21c994ef68e4496f56d12b4c9, State: Initialized, Role: FOLLOWER
I20260927 10:48:33.613044 11152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42861 } }
I20260927 10:48:33.613186 11152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:33.613269 11152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:33.613358 11152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 10:48:33 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:33.614226 11152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42861 } }
I20260927 10:48:33.614537 11152 leader_election.cc:302] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a7e85e21c994ef68e4496f56d12b4c9; no voters:
I20260927 10:48:33.614876 11152 leader_election.cc:288] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:33.615006 11157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:33.615180 11157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [term 1 LEADER]: Becoming Leader. State: Replica: 6a7e85e21c994ef68e4496f56d12b4c9, State: Running, Role: LEADER
I20260927 10:48:33.615439 11157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42861 } }
I20260927 10:48:33.616035 11152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:33.617095 11159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a7e85e21c994ef68e4496f56d12b4c9. Latest consensus state: current_term: 1 leader_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42861 } } }
I20260927 10:48:33.617247 11159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:33.617094 11158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7e85e21c994ef68e4496f56d12b4c9" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42861 } } }
I20260927 10:48:33.617336 11158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:33.617604 11166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:33.620087 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:33.606578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.9.39.190:42861)
0927 10:48:33.607017 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:33.607031 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:33.607062 (+ 31us) server_negotiation.cc:407] Connection header received
0927 10:48:33.607849 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:33.607874 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:33.608234 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:33.608547 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:33.609340 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.610279 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.611225 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.611689 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.611853 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:33.614397 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:33.614401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:33.614413 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:33.614435 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:33.616465 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:33.616993 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:33.616998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:33.617004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:33.617159 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:33.617475 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:33.617480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:33.617482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:33.617980 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:33.618211 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:33.618886 (+ 675us) server_negotiation.cc:299] Negotiation successful
0927 10:48:33.619174 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":176,"threads_started":1}
I20260927 10:48:33.620661 11166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:33.625602 11166 catalog_manager.cc:1347] Generated new cluster ID: e2b49db6e7c445e1ba0e1df0057c51fd
I20260927 10:48:33.625689 11166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:33.641376 11166 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:33.642838 11166 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:33.648638 11166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Generated new TSK 0
I20260927 10:48:33.649240 11166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:33.677819 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42861
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:33.783988 11178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:33.784215 11178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:33.784305 11178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:33.788539 11178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:33.788596 11178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:33.788637 11178 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 10:48:33.788704 11178 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:48:33.788775 11178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:33.793879 11178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:33.794862 11178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:33.795805 11178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:33.804491 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:33.804515 11183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:33.804549 11184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:33.805308 11178 server_base.cc:1034] running on GCE node
I20260927 10:48:33.805680 11178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:33.806316 11178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:33.807489 11178 hybrid_clock.cc:648] HybridClock initialized: now 1790506113807461 us; error 48 us; skew 500 ppm
Sep 27 10:48:33 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506113, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:33.810048 11178 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:33.810937 11178 webserver.cc:460] Webserver started at http://127.9.39.129:40145/ using document root <none> and password file <none>
I20260927 10:48:33.811481 11178 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:33.811565 11178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:33.811777 11178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:33.813686 11178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "cc31fe1a65594e738b80676f16ac3f57"
format_stamp: "Formatted at 2026-09-27 10:48:33 on dist-test-slave-p9q6"
server_key: "c40ddced5b2c254db47dee97099407df"
server_key_iv: "40b2e48bdce18a52db1552e435e2345d"
server_key_version: "encryptionkey@0"
I20260927 10:48:33.814260 11178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "cc31fe1a65594e738b80676f16ac3f57"
format_stamp: "Formatted at 2026-09-27 10:48:33 on dist-test-slave-p9q6"
server_key: "c40ddced5b2c254db47dee97099407df"
server_key_iv: "40b2e48bdce18a52db1552e435e2345d"
server_key_version: "encryptionkey@0"
I20260927 10:48:33.817978 11178 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 10:48:33.820264 11193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:33.821336 11178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 10:48:33.821460 11178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "cc31fe1a65594e738b80676f16ac3f57"
format_stamp: "Formatted at 2026-09-27 10:48:33 on dist-test-slave-p9q6"
server_key: "c40ddced5b2c254db47dee97099407df"
server_key_iv: "40b2e48bdce18a52db1552e435e2345d"
server_key_version: "encryptionkey@0"
I20260927 10:48:33.821558 11178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:33.841455 11178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:33.844046 11178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:33.844238 11178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:33.845526 11178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:33.845597 11178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:33.845669 11178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:33.845712 11178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:48:33 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506113, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:33.856287 11178 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:40085
I20260927 10:48:33.856375 11308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:40085 every 8 connection(s)
I20260927 10:48:33.857339 11178 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:33.859023 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:33.846294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:44393 (local address 127.9.39.190:42861)
0927 10:48:33.846410 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:33.846414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:33.847418 (+ 1004us) server_negotiation.cc:407] Connection header received
0927 10:48:33.848424 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:33.848431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:33.848494 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:33.848558 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:33.849723 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.850455 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.851831 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.852077 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.852197 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:33.854665 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:33.854668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:33.854671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:33.854711 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:33.856323 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:33.856873 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:33.856875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:33.856877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:33.856948 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:33.857290 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:33.857293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:33.857293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:33.857522 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:33.857605 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:33.858819 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0927 10:48:33.858913 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:48:33.859170 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:33.846822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.9.39.129:44393)
0927 10:48:33.847288 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:33.847325 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:33.848188 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:33.848875 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:33.848880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:33.849166 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:33.849572 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:33.849583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.850573 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.850578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:33.851721 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:33.851735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.852230 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.852234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:33.852385 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:33.852982 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:33.853002 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:33.854523 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:33.856452 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:33.856457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:33.856469 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:33.856778 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:33.857041 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:33.857044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:33.857046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:33.857203 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:33.857621 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:33.857628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:33.857928 (+ 300us) client_negotiation.cc:769] Sending connection context
0927 10:48:33.858124 (+ 196us) client_negotiation.cc:240] Negotiation successful
0927 10:48:33.858415 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":137,"threads_started":1}
I20260927 10:48:33.860251 11309 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42861
I20260927 10:48:33.860390 11309 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:33.860805 11309 heartbeater.cc:511] Master 127.9.39.190:42861 requested a full tablet report, sending...
I20260927 10:48:33.862221 11106 ts_manager.cc:194] Registered new tserver with Master: cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085)
I20260927 10:48:33.863353 11106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:44393
I20260927 10:48:33.863737 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11178
I20260927 10:48:33.863848 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:33.864133 9374 external_mini_cluster.cc:1428] Setting key ee27f6c771060f679e57c4bd23be2df5
I20260927 10:48:33.869393 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:33.862310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.9.39.129:37555)
0927 10:48:33.862541 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:33.862552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:33.862654 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:33.862906 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:33.862908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:33.863081 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:33.863162 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:33.863167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.864025 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.864028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:33.864810 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:33.864821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.865464 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.865467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:33.865553 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:33.866034 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:33.866044 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:33.866341 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:33.868032 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:33.868038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:33.868040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:33.868290 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:33.868518 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:33.868521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:33.868522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:33.868615 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:33.869035 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:33.869037 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:33.869100 (+ 63us) client_negotiation.cc:769] Sending connection context
0927 10:48:33.869175 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 10:48:33.869265 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260927 10:48:33.869592 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:33.862252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:37555 (local address 127.9.39.190:42861)
0927 10:48:33.862379 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:33.862384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:33.862620 (+ 236us) server_negotiation.cc:407] Connection header received
0927 10:48:33.862765 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:33.862768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:33.862806 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:33.862893 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:33.863302 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.863886 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.864938 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:33.865256 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:33.865328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:33.866473 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:33.866479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:33.866485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:33.866514 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:33.867913 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:33.868379 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:33.868381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:33.868382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:33.868443 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:33.868720 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:33.868726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:33.868730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:33.868946 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:33.869044 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:33.869256 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 10:48:33.869370 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:33.891407 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42861
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:33.995497 11318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:33.995783 11318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:33.995862 11318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:33.999926 11318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:34.000000 11318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:34.000068 11318 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 10:48:34.000110 11318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:48:34.000170 11318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:34.005138 11318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:34.006119 11318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:34.007218 11318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:34.016141 11324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:34.016161 11326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:34.016141 11323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:34.016506 11318 server_base.cc:1034] running on GCE node
I20260927 10:48:34.016971 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:34.017495 11318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:34.018644 11318 hybrid_clock.cc:648] HybridClock initialized: now 1790506114018616 us; error 39 us; skew 500 ppm
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506114, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:34.021252 11318 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:34.021993 11318 webserver.cc:460] Webserver started at http://127.9.39.130:44295/ using document root <none> and password file <none>
I20260927 10:48:34.022455 11318 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:34.022506 11318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:34.022650 11318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:34.024600 11318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "48f81b5ed713418485d4194140dab6ee"
format_stamp: "Formatted at 2026-09-27 10:48:34 on dist-test-slave-p9q6"
server_key: "3d76592327cf10fd3dc9ae89416fbcaf"
server_key_iv: "9b7ed586ba7d8478b942cc0e70f1b9ba"
server_key_version: "encryptionkey@0"
I20260927 10:48:34.025103 11318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "48f81b5ed713418485d4194140dab6ee"
format_stamp: "Formatted at 2026-09-27 10:48:34 on dist-test-slave-p9q6"
server_key: "3d76592327cf10fd3dc9ae89416fbcaf"
server_key_iv: "9b7ed586ba7d8478b942cc0e70f1b9ba"
server_key_version: "encryptionkey@0"
I20260927 10:48:34.028594 11318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:48:34.030784 11333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:34.031981 11318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:34.032094 11318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "48f81b5ed713418485d4194140dab6ee"
format_stamp: "Formatted at 2026-09-27 10:48:34 on dist-test-slave-p9q6"
server_key: "3d76592327cf10fd3dc9ae89416fbcaf"
server_key_iv: "9b7ed586ba7d8478b942cc0e70f1b9ba"
server_key_version: "encryptionkey@0"
I20260927 10:48:34.032213 11318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:34.063565 11318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:34.067065 11318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:34.067267 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:34.068703 11318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:34.068776 11318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:34.068843 11318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:34.068895 11318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506114, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:34.079864 11318 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:41747
I20260927 10:48:34.079958 11448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:41747 every 8 connection(s)
I20260927 10:48:34.080945 11318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:34.083418 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.069603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:49511 (local address 127.9.39.190:42861)
0927 10:48:34.069725 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.069729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.070806 (+ 1077us) server_negotiation.cc:407] Connection header received
0927 10:48:34.071882 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.071889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.071961 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.072070 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:34.073695 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.074734 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.075962 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.076205 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.076323 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.079633 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:34.079637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:34.079642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:34.079674 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:34.081236 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.081816 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.081821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.081823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.081910 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.082249 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.082252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.082254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.082440 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:34.082550 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.083116 (+ 566us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.083250 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:48:34.084141 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.070168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.9.39.130:49511)
0927 10:48:34.070623 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.070662 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.071639 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.072300 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.072307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.072820 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:34.073520 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.073542 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.074871 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.074875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.075831 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.075841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.076373 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.076377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.076581 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.077752 (+ 1171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:34.077774 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:34.079458 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:34.081368 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.081374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.081387 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.081706 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.082007 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.082010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.082012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.082147 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.082546 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:34.082553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:34.082875 (+ 322us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.083073 (+ 198us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.083374 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260927 10:48:34.085273 11449 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42861
I20260927 10:48:34.085420 11449 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:34.085930 11449 heartbeater.cc:511] Master 127.9.39.190:42861 requested a full tablet report, sending...
I20260927 10:48:34.086861 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11318
I20260927 10:48:34.086978 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:34.086956 11100 ts_manager.cc:194] Registered new tserver with Master: 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747)
I20260927 10:48:34.087327 9374 external_mini_cluster.cc:1428] Setting key 175c73090de53ad717e384a36b459685
I20260927 10:48:34.087528 11100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:49511
I20260927 10:48:34.095188 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.088126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:39783 (local address 127.9.39.190:42861)
0927 10:48:34.088246 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.088250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.088711 (+ 461us) server_negotiation.cc:407] Connection header received
0927 10:48:34.088847 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.088850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.088894 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.088977 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:34.089379 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.089976 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.090835 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.091061 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.091162 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.092223 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:34.092226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:34.092228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:34.092252 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:34.093622 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.094049 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.094053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.094055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.094128 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.094404 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.094408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.094410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.094590 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:34.094722 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.094956 (+ 234us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.095056 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 10:48:34.095197 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.088002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.9.39.130:39783)
0927 10:48:34.088629 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.088641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.088752 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.088991 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.088994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.089187 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:34.089270 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.089275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.090106 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.090109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.090673 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.090680 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.091180 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.091185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.091275 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.091771 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:34.091782 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:34.092081 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:34.093764 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.093768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.093771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.093956 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.094226 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.094229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.094231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.094308 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.094716 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:34.094719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:34.094803 (+ 84us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.094899 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.095017 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":480,"threads_started":1}
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:34.113384 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42861
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:34.224958 11458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:34.225250 11458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:34.225298 11458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:34.229386 11458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:34.229442 11458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:34.229482 11458 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 10:48:34.229496 11458 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:48:34.229527 11458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:34.234979 11458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:34.235987 11458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:34.236909 11458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:34.245848 11464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:34.245857 11463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:34.245880 11466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:34.246148 11458 server_base.cc:1034] running on GCE node
I20260927 10:48:34.246508 11458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:34.247121 11458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:34.248287 11458 hybrid_clock.cc:648] HybridClock initialized: now 1790506114248270 us; error 29 us; skew 500 ppm
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506114, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:34.250777 11458 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:34.251565 11458 webserver.cc:460] Webserver started at http://127.9.39.131:42817/ using document root <none> and password file <none>
I20260927 10:48:34.252091 11458 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:34.252149 11458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:34.252398 11458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:34.254212 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "d9aa07b1b1da436199a6bea42eb10e88"
format_stamp: "Formatted at 2026-09-27 10:48:34 on dist-test-slave-p9q6"
server_key: "430025e653b979ea511735adb3ed0930"
server_key_iv: "22263ee8bf185842471285c377573bc9"
server_key_version: "encryptionkey@0"
I20260927 10:48:34.254815 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "d9aa07b1b1da436199a6bea42eb10e88"
format_stamp: "Formatted at 2026-09-27 10:48:34 on dist-test-slave-p9q6"
server_key: "430025e653b979ea511735adb3ed0930"
server_key_iv: "22263ee8bf185842471285c377573bc9"
server_key_version: "encryptionkey@0"
I20260927 10:48:34.258157 11458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:34.260552 11473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:34.261915 11458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:34.262041 11458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "d9aa07b1b1da436199a6bea42eb10e88"
format_stamp: "Formatted at 2026-09-27 10:48:34 on dist-test-slave-p9q6"
server_key: "430025e653b979ea511735adb3ed0930"
server_key_iv: "22263ee8bf185842471285c377573bc9"
server_key_version: "encryptionkey@0"
I20260927 10:48:34.262151 11458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:34.279712 11458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:34.282279 11458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:34.282469 11458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:34.283797 11458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:34.283841 11458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:34.283900 11458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:34.283952 11458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506114, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:34.294966 11458 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:33299
I20260927 10:48:34.295293 11588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:33299 every 8 connection(s)
I20260927 10:48:34.295820 11458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:34.297749 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.284638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:55705 (local address 127.9.39.190:42861)
0927 10:48:34.284814 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.284820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.285488 (+ 668us) server_negotiation.cc:407] Connection header received
0927 10:48:34.286501 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.286504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.286561 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.286622 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:34.288104 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.288984 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.290231 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.290458 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.290576 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.294145 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:34.294149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:34.294151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:34.294174 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:34.295756 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.296256 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.296261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.296264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.296346 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.296731 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.296736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.296738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.296915 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:34.297018 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.297471 (+ 453us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.297585 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 10:48:34.298425 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.284952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.9.39.131:55705)
0927 10:48:34.285357 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.285403 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.286265 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.287055 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.287061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.287394 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:34.287897 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.287909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.289130 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.289133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.290100 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.290109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.290649 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.290653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.290867 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.291541 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:34.291562 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:34.293976 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:34.295870 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.295876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.295889 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.296154 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.296473 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.296479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.296481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.296626 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.297025 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:34.297033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:34.297247 (+ 214us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.297418 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.297709 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":123,"threads_started":1}
I20260927 10:48:34.298911 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11458
I20260927 10:48:34.299024 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:34.299335 9374 external_mini_cluster.cc:1428] Setting key 692a0fcc799353c07b3d1f8799c7231a
I20260927 10:48:34.299809 11589 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42861
I20260927 10:48:34.299955 11589 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:34.300462 11589 heartbeater.cc:511] Master 127.9.39.190:42861 requested a full tablet report, sending...
I20260927 10:48:34.301379 11106 ts_manager.cc:194] Registered new tserver with Master: d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.302148 11106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:55705
I20260927 10:48:34.307758 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.301775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:38599 (local address 127.9.39.190:42861)
0927 10:48:34.301886 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.301890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.302008 (+ 118us) server_negotiation.cc:407] Connection header received
0927 10:48:34.302094 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.302096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.302128 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.302191 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:34.302625 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.303361 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.304153 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.304348 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.304425 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.305336 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:34.305338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:34.305340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:34.305354 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:34.306497 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.306915 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.306917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.306918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.306982 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.307181 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.307183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.307184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.307312 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:34.307418 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.307576 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.307652 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:48:34.307762 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.301663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.9.39.131:38599)
0927 10:48:34.301926 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.301940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.302022 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.302224 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.302229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.302415 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:34.302525 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.302532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.303511 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.303514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.304057 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.304064 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.304438 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.304440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.304500 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.304949 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:34.304960 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:34.305234 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:34.306586 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.306588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.306589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.306842 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.307055 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.307057 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.307058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.307125 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.307402 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:34.307404 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:34.307467 (+ 63us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.307540 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.307628 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260927 10:48:34.312620 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:34.337368 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.328879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.9.39.190:42861)
0927 10:48:34.328992 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.328995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.329006 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:48:34.329038 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.329040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.329073 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.329140 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:34.329758 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.330231 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.331066 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.331283 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.331384 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.334064 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:34.334068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:34.334071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:34.334096 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:34.335569 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.336044 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.336050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.336052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.336169 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.336490 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.336494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.336496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.336679 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:34.336832 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.337072 (+ 240us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.337186 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:48:34.340129 11106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:34.343110 11106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:34.358543 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.351635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40085 (local address 127.0.0.1:40648)
0927 10:48:34.352550 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.352595 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.352730 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.353909 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.353915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.353937 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.354064 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.354070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.355531 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.355533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.356942 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.356953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.358018 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.358021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.358135 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.358178 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.358263 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.358390 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260927 10:48:34.358963 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.351754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41747 (local address 127.0.0.1:40672)
0927 10:48:34.352989 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.353004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.353090 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.354513 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.354516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.354534 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.354635 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.354643 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.356083 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.356090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.356998 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.357007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.358422 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.358425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.358516 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.358540 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.358756 (+ 216us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.358827 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"thread_start_us":101,"threads_started":1}
I20260927 10:48:34.359117 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.352657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33299 (local address 127.0.0.1:41554)
0927 10:48:34.353473 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.353485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.353567 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.355240 (+ 1673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.355243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.355258 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.355331 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.355335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.356686 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.356689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.357563 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.357570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.358652 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.358658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.358783 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.358810 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.358870 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.358930 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":23,"thread_start_us":78,"threads_started":1}
I20260927 10:48:34.359172 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.352933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.9.39.131:33299)
0927 10:48:34.353263 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.353275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.353974 (+ 699us) server_negotiation.cc:407] Connection header received
0927 10:48:34.354048 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.354052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.354239 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.355117 (+ 878us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.355439 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.356566 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.357687 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.358534 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.358647 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.358863 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.358913 (+ 50us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.359007 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
I20260927 10:48:34.359611 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.353718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.9.39.130:41747)
0927 10:48:34.354041 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.354052 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.354074 (+ 22us) server_negotiation.cc:407] Connection header received
0927 10:48:34.354144 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.354149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.354351 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.354799 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.354927 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.355970 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.357124 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.358301 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.358399 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.358553 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.359367 (+ 814us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.359450 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":108,"threads_started":1}
I20260927 10:48:34.359880 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.352475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.9.39.129:40085)
0927 10:48:34.353398 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.353412 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.353435 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:48:34.353515 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.353523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.353736 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.354205 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.354353 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.355420 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.357087 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.357893 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.359395 (+ 1502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.359540 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.359584 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.359759 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":739,"threads_started":1}
I20260927 10:48:34.362074 11523 tablet_service.cc:1467] Processing CreateTablet for tablet f9e4559974db41e0ab3820ac16f0501a (DEFAULT_TABLE table=test-table [id=07db5d3ccc6745f7be9467c87c0cd7b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:34.362217 11383 tablet_service.cc:1467] Processing CreateTablet for tablet f9e4559974db41e0ab3820ac16f0501a (DEFAULT_TABLE table=test-table [id=07db5d3ccc6745f7be9467c87c0cd7b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:34.363116 11523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e4559974db41e0ab3820ac16f0501a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.363157 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e4559974db41e0ab3820ac16f0501a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.365127 11243 tablet_service.cc:1467] Processing CreateTablet for tablet f9e4559974db41e0ab3820ac16f0501a (DEFAULT_TABLE table=test-table [id=07db5d3ccc6745f7be9467c87c0cd7b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:34.366037 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e4559974db41e0ab3820ac16f0501a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.369509 11612 tablet_bootstrap.cc:492] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Bootstrap starting.
I20260927 10:48:34.370821 11613 tablet_bootstrap.cc:492] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Bootstrap starting.
I20260927 10:48:34.371829 11612 tablet_bootstrap.cc:654] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.372674 11612 log.cc:824] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:34.372836 11613 tablet_bootstrap.cc:654] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.373349 11614 tablet_bootstrap.cc:492] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Bootstrap starting.
I20260927 10:48:34.373625 11613 log.cc:824] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:34.374766 11612 tablet_bootstrap.cc:492] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: No bootstrap required, opened a new log
I20260927 10:48:34.374967 11612 ts_tablet_manager.cc:1397] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:34.375193 11614 tablet_bootstrap.cc:654] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.375468 11613 tablet_bootstrap.cc:492] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: No bootstrap required, opened a new log
I20260927 10:48:34.375635 11613 ts_tablet_manager.cc:1397] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:48:34.376330 11614 log.cc:824] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:34.378085 11614 tablet_bootstrap.cc:492] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: No bootstrap required, opened a new log
I20260927 10:48:34.378254 11614 ts_tablet_manager.cc:1397] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:48:34.378155 11612 raft_consensus.cc:348] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.378432 11612 raft_consensus.cc:374] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.378484 11612 raft_consensus.cc:729] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9aa07b1b1da436199a6bea42eb10e88, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.379089 11612 consensus_queue.cc:260] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.379315 11613 raft_consensus.cc:348] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.379556 11613 raft_consensus.cc:374] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.379611 11613 raft_consensus.cc:729] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.379967 11612 ts_tablet_manager.cc:1428] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:48:34.380023 11613 consensus_queue.cc:260] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.380331 11589 heartbeater.cc:503] Master 127.9.39.190:42861 was elected leader, sending a full tablet report...
I20260927 10:48:34.380648 11449 heartbeater.cc:503] Master 127.9.39.190:42861 was elected leader, sending a full tablet report...
I20260927 10:48:34.380914 11613 ts_tablet_manager.cc:1428] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:48:34.381817 11614 raft_consensus.cc:348] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.382053 11614 raft_consensus.cc:374] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.382104 11614 raft_consensus.cc:729] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.382485 11614 consensus_queue.cc:260] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.383209 11309 heartbeater.cc:503] Master 127.9.39.190:42861 was elected leader, sending a full tablet report...
I20260927 10:48:34.383385 11614 ts_tablet_manager.cc:1428] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260927 10:48:34.547088 11590 tablet.cc:2369] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:34.570331 11620 raft_consensus.cc:482] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:34.570551 11620 raft_consensus.cc:504] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.571758 11620 leader_election.cc:288] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.575896 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.572312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:50757 (local address 127.9.39.131:33299)
0927 10:48:34.572477 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.572482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.572583 (+ 101us) server_negotiation.cc:407] Connection header received
0927 10:48:34.572731 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.572733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.572774 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.572850 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.573134 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.573734 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.574883 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.575286 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.575429 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.575494 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.575685 (+ 191us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.575749 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 10:48:34.575938 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.572226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:32945 (local address 127.9.39.130:41747)
0927 10:48:34.572368 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.572373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.572637 (+ 264us) server_negotiation.cc:407] Connection header received
0927 10:48:34.572788 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.572791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.572842 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.572918 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.573206 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.573786 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.574927 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.575303 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.575444 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.575511 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.575736 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.575816 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 10:48:34.575959 11621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.572096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33299 (local address 127.9.39.129:50757)
0927 10:48:34.572490 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.572502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.572599 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.572871 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.572873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.572888 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.573027 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.573032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.573828 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.573833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.574779 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.574788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.575422 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.575425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.575567 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.575584 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.575672 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.575739 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":50,"thread_start_us":151,"threads_started":1}
I20260927 10:48:34.576332 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.572096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41747 (local address 127.9.39.129:32945)
0927 10:48:34.572534 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.572564 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.572678 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.572941 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.572944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.572958 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.573099 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.573105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.573888 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.573894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.574820 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.574829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.575438 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.575442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.575618 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.575634 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.575722 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.575803 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":116,"thread_start_us":104,"threads_started":1}
I20260927 10:48:34.576707 11543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e4559974db41e0ab3820ac16f0501a" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88" is_pre_election: true
I20260927 10:48:34.576946 11403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e4559974db41e0ab3820ac16f0501a" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f81b5ed713418485d4194140dab6ee" is_pre_election: true
I20260927 10:48:34.577013 11543 raft_consensus.cc:2454] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 0.
I20260927 10:48:34.577200 11403 raft_consensus.cc:2454] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 0.
I20260927 10:48:34.577500 11194 leader_election.cc:302] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc31fe1a65594e738b80676f16ac3f57, d9aa07b1b1da436199a6bea42eb10e88; no voters:
I20260927 10:48:34.577800 11620 raft_consensus.cc:2781] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:34.577872 11620 raft_consensus.cc:482] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:34.577921 11620 raft_consensus.cc:3037] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.579028 11620 raft_consensus.cc:504] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.579411 11620 leader_election.cc:288] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 election: Requested vote from peers 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.579721 11403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e4559974db41e0ab3820ac16f0501a" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f81b5ed713418485d4194140dab6ee"
I20260927 10:48:34.579839 11543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e4559974db41e0ab3820ac16f0501a" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88"
I20260927 10:48:34.579836 11403 raft_consensus.cc:3037] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.579960 11543 raft_consensus.cc:3037] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.580986 11403 raft_consensus.cc:2454] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 1.
I20260927 10:48:34.581331 11195 leader_election.cc:302] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, cc31fe1a65594e738b80676f16ac3f57; no voters:
I20260927 10:48:34.581475 11620 raft_consensus.cc:2781] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:34.581697 11620 raft_consensus.cc:686] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 1 LEADER]: Becoming Leader. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Running, Role: LEADER
I20260927 10:48:34.581808 11543 raft_consensus.cc:2454] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 1.
I20260927 10:48:34.582027 11620 consensus_queue.cc:237] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
W20260927 10:48:34.582212 11450 tablet.cc:2369] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:34.585016 11105 catalog_manager.cc:5495] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "cc31fe1a65594e738b80676f16ac3f57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } health_report { overall_health: UNKNOWN } } }
W20260927 10:48:34.608671 11310 tablet.cc:2369] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 10:48:34 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506113, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:34.664407 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.656341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.9.39.190:42861)
0927 10:48:34.656469 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.656474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.656633 (+ 159us) server_negotiation.cc:407] Connection header received
0927 10:48:34.656737 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.656740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.656778 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.656845 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:34.657430 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.657914 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.658741 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.658955 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.659033 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.661427 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:34.661430 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:34.661432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:34.661451 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:34.662607 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.663068 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.663075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.663077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.663148 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:34.663418 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:34.663421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:34.663423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:34.663579 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:34.663796 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.664088 (+ 292us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.664248 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":50}
I20260927 10:48:34.664407 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.656292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42861 (local address 127.0.0.1:51916)
0927 10:48:34.656550 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.656563 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.656663 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.656882 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.656885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.657253 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:34.657337 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.657342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.658004 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.658006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.658581 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.658588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.659048 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.659050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.659111 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.659680 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:34.659691 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:34.661293 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:34.662728 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.662740 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.662746 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.662971 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.663248 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:34.663250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:34.663252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:34.663326 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:34.663794 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:34.663802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:34.663918 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.664045 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.664193 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
I20260927 10:48:34.668221 11105 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:51916:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 10:48:34.668839 11105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:34.673363 11243 tablet_service.cc:1467] Processing CreateTablet for tablet 2acd80ddbc3745958f6f1b0fe8afb4a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78ebdda3866f422fad59b473dbf2b3bf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:48:34.673436 11383 tablet_service.cc:1467] Processing CreateTablet for tablet 2acd80ddbc3745958f6f1b0fe8afb4a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78ebdda3866f422fad59b473dbf2b3bf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:48:34.673568 11523 tablet_service.cc:1467] Processing CreateTablet for tablet 2acd80ddbc3745958f6f1b0fe8afb4a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78ebdda3866f422fad59b473dbf2b3bf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:48:34.674118 11523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2acd80ddbc3745958f6f1b0fe8afb4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.674118 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2acd80ddbc3745958f6f1b0fe8afb4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.674151 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2acd80ddbc3745958f6f1b0fe8afb4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.676973 11612 tablet_bootstrap.cc:492] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: Bootstrap starting.
I20260927 10:48:34.676973 11614 tablet_bootstrap.cc:492] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: Bootstrap starting.
I20260927 10:48:34.676983 11613 tablet_bootstrap.cc:492] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: Bootstrap starting.
I20260927 10:48:34.677822 11613 tablet_bootstrap.cc:654] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.678026 11612 tablet_bootstrap.cc:654] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.678398 11614 tablet_bootstrap.cc:654] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.679087 11613 tablet_bootstrap.cc:492] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: No bootstrap required, opened a new log
I20260927 10:48:34.679183 11613 ts_tablet_manager.cc:1397] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:34.679399 11612 tablet_bootstrap.cc:492] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: No bootstrap required, opened a new log
I20260927 10:48:34.679474 11612 ts_tablet_manager.cc:1397] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:34.679739 11614 tablet_bootstrap.cc:492] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: No bootstrap required, opened a new log
I20260927 10:48:34.679809 11614 ts_tablet_manager.cc:1397] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:34.679821 11613 raft_consensus.cc:348] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.679908 11613 raft_consensus.cc:374] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.679927 11613 raft_consensus.cc:729] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.679919 11612 raft_consensus.cc:348] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.680020 11612 raft_consensus.cc:374] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.680052 11612 raft_consensus.cc:729] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9aa07b1b1da436199a6bea42eb10e88, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.680058 11613 consensus_queue.cc:260] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.680184 11612 consensus_queue.cc:260] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.680251 11614 raft_consensus.cc:348] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.680317 11613 ts_tablet_manager.cc:1428] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 10:48:34.680344 11614 raft_consensus.cc:374] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.680377 11614 raft_consensus.cc:729] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.680459 11612 ts_tablet_manager.cc:1428] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:34.680493 11614 consensus_queue.cc:260] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.680466 11618 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.680621 11618 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.680763 11620 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.680874 11620 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.681095 11614 ts_tablet_manager.cc:1428] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:34.681200 11619 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.681317 11619 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.695448 11619 raft_consensus.cc:482] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:34.695567 11619 raft_consensus.cc:504] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.696704 11619 leader_election.cc:288] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.701370 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.697000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:59897 (local address 127.9.39.129:40085)
0927 10:48:34.697155 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.697160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.697289 (+ 129us) server_negotiation.cc:407] Connection header received
0927 10:48:34.697426 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.697429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.697492 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.697555 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.697852 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.698629 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.700446 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.700879 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.700969 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.701006 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.701203 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.701275 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:48:34.701392 11634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.696883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40085 (local address 127.9.39.130:59897)
0927 10:48:34.697172 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.697187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.697315 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.697572 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.697575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.697598 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.697747 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.697753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.698796 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.698799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.700318 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.700329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.700986 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.700989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.701067 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.701085 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.701165 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.701237 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":68,"thread_start_us":105,"threads_started":1}
I20260927 10:48:34.702028 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.697274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33299 (local address 127.9.39.130:40421)
0927 10:48:34.697576 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.697588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.697668 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.697892 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.697894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.697908 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:34.697991 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.697995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.698889 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.698892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.700318 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.700329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.701682 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.701685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.701784 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.701804 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.701875 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.701919 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":153,"thread_start_us":81,"threads_started":1}
I20260927 10:48:34.702025 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd80ddbc3745958f6f1b0fe8afb4a2" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc31fe1a65594e738b80676f16ac3f57" is_pre_election: true
I20260927 10:48:34.702104 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.697377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:40421 (local address 127.9.39.131:33299)
0927 10:48:34.697488 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.697493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.697676 (+ 183us) server_negotiation.cc:407] Connection header received
0927 10:48:34.697745 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.697747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.697782 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.697871 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:34.698087 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.698810 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.701051 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.701413 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.701554 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.701611 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.701912 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.701992 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 10:48:34.702271 11263 raft_consensus.cc:2454] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 0.
I20260927 10:48:34.702658 11334 leader_election.cc:302] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, cc31fe1a65594e738b80676f16ac3f57; no voters:
I20260927 10:48:34.702827 11543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd80ddbc3745958f6f1b0fe8afb4a2" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88" is_pre_election: true
I20260927 10:48:34.702950 11543 raft_consensus.cc:2454] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 0.
I20260927 10:48:34.702962 11619 raft_consensus.cc:2781] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:34.703024 11619 raft_consensus.cc:482] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:34.703073 11619 raft_consensus.cc:3037] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.704057 11619 raft_consensus.cc:504] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.704370 11619 leader_election.cc:288] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 election: Requested vote from peers cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.704701 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd80ddbc3745958f6f1b0fe8afb4a2" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc31fe1a65594e738b80676f16ac3f57"
I20260927 10:48:34.704821 11263 raft_consensus.cc:3037] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.704823 11543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd80ddbc3745958f6f1b0fe8afb4a2" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88"
I20260927 10:48:34.704929 11543 raft_consensus.cc:3037] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.705752 11263 raft_consensus.cc:2454] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 1.
I20260927 10:48:34.705771 11543 raft_consensus.cc:2454] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 1.
I20260927 10:48:34.706019 11334 leader_election.cc:302] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, d9aa07b1b1da436199a6bea42eb10e88; no voters:
I20260927 10:48:34.706180 11619 raft_consensus.cc:2781] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:34.706401 11619 raft_consensus.cc:686] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [term 1 LEADER]: Becoming Leader. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Running, Role: LEADER
I20260927 10:48:34.706809 11619 consensus_queue.cc:237] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.708854 11636 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: TxnStatusTablet state changed. Reason: New leader 48f81b5ed713418485d4194140dab6ee. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.708959 11636 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:48:34.709245 11638 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 10:48:34.709460 11638 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 10:48:34.710079 11106 catalog_manager.cc:5495] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 48f81b5ed713418485d4194140dab6ee (127.9.39.130). New cstate: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:34.724349 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.720584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.9.39.130:41747)
0927 10:48:34.720726 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.720731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.720745 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:48:34.720905 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.720908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.720967 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.721065 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:34.721307 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.722063 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.722847 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.723215 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.723306 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.723542 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.723982 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.724087 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.724171 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.724224 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 10:48:34.724370 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.720477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41747 (local address 127.0.0.1:40682)
0927 10:48:34.720628 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.720645 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.720759 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.721078 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.721080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.721085 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:34.721191 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.721195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.722168 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.722170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.722750 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.722757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.723319 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.723321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.723388 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.723427 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.724067 (+ 640us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.724077 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.724119 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.724220 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 10:48:34.729467 11263 raft_consensus.cc:1261] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Refusing update from remote peer 48f81b5ed713418485d4194140dab6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:34.729532 11543 raft_consensus.cc:1261] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Refusing update from remote peer 48f81b5ed713418485d4194140dab6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:34.730118 11636 consensus_queue.cc:1035] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:34.730353 11619 consensus_queue.cc:1035] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:34.733695 11618 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: TxnStatusTablet state changed. Reason: New leader 48f81b5ed713418485d4194140dab6ee. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.733819 11618 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.733280 11620 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: TxnStatusTablet state changed. Reason: New leader 48f81b5ed713418485d4194140dab6ee. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.734817 11620 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.737802 11619 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.737906 11619 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:48:34.738329 11620 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.738425 11620 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P cc31fe1a65594e738b80676f16ac3f57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.739465 11636 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.739573 11636 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P 48f81b5ed713418485d4194140dab6ee: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:48:34.740595 11618 tablet_replica.cc:439] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } } }
I20260927 10:48:34.740697 11618 tablet_replica.cc:442] T 2acd80ddbc3745958f6f1b0fe8afb4a2 P d9aa07b1b1da436199a6bea42eb10e88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:48:34.740868 11642 mvcc.cc:205] Tried to move back new op lower bound from 7333913045925675008 to 7333913045841764352. Current Snapshot: MvccSnapshot[applied={T|T < 7333913045925675008}]
I20260927 10:48:34.751147 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.747121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.9.39.129:40085)
0927 10:48:34.747241 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.747244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.747275 (+ 31us) server_negotiation.cc:407] Connection header received
0927 10:48:34.747438 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.747440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.747487 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.747565 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:34.747825 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.748300 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.749521 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.749926 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.750017 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.750327 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.750759 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.750856 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.750960 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.751006 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:48:34.761618 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.757931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:41029 (local address 127.9.39.130:41747)
0927 10:48:34.758048 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.758052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.758197 (+ 145us) server_negotiation.cc:407] Connection header received
0927 10:48:34.758295 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.758298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.758348 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.758425 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:34.758634 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.759418 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.760360 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.760704 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.760805 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.761053 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.761141 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.761223 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.761440 (+ 217us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.761490 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:48:34.761623 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.757837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41747 (local address 127.9.39.129:41029)
0927 10:48:34.758105 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.758118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.758209 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.758438 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.758440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.758445 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:34.758544 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.758547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.759532 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.759535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.760269 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.760279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.760808 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.760811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.760877 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.760917 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.761298 (+ 381us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.761310 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.761426 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.761494 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":102,"threads_started":1}
I20260927 10:48:34.769261 11543 raft_consensus.cc:1261] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Refusing update from remote peer cc31fe1a65594e738b80676f16ac3f57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:34.769407 11403 raft_consensus.cc:1261] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Refusing update from remote peer cc31fe1a65594e738b80676f16ac3f57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:34.769762 11623 consensus_queue.cc:1035] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:34.769930 11620 consensus_queue.cc:1035] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:34.779394 11641 mvcc.cc:205] Tried to move back new op lower bound from 7333913046090207232 to 7333913045331107840. Current Snapshot: MvccSnapshot[applied={T|T < 7333913046090207232}]
I20260927 10:48:34.792766 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.788415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:48485 (local address 127.9.39.129:40085)
0927 10:48:34.788540 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:34.788552 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:34.788910 (+ 358us) server_negotiation.cc:407] Connection header received
0927 10:48:34.789022 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:34.789025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:34.789077 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:34.789170 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:34.789460 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.790257 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.791607 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.791860 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.791983 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.792314 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.792398 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.792473 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:34.792562 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 10:48:34.792618 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:48:34.792805 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:34.788305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40085 (local address 127.9.39.130:48485)
0927 10:48:34.788816 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:34.788832 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:34.788937 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:34.789179 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:34.789182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:34.789205 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:34.789332 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.789338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.790400 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.790404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.791474 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:34.791485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:34.791994 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:34.791998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:34.792092 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:34.792169 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:34.792476 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:34.792482 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 10:48:34.792546 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 10:48:34.792635 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":113,"threads_started":1}
I20260927 10:48:34.816803 11100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:34.817013 11100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:34.819528 11100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Sending DeleteTablet for 3 replicas of tablet f9e4559974db41e0ab3820ac16f0501a
I20260927 10:48:34.820135 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e4559974db41e0ab3820ac16f0501a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:34 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40648
I20260927 10:48:34.820181 11383 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e4559974db41e0ab3820ac16f0501a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:34 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40672
I20260927 10:48:34.820683 11662 tablet_replica.cc:330] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: stopping tablet replica
I20260927 10:48:34.820931 11523 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e4559974db41e0ab3820ac16f0501a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:34 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:41554
I20260927 10:48:34.821003 11662 raft_consensus.cc:2229] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:34.821424 11662 raft_consensus.cc:2258] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:34.822979 11663 tablet_replica.cc:330] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: stopping tablet replica
I20260927 10:48:34.823247 11663 raft_consensus.cc:2229] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:34.823397 11663 raft_consensus.cc:2258] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:34.823550 11100 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:34.823935 11100 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:34.824744 11663 ts_tablet_manager.cc:1905] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:34.827836 11663 ts_tablet_manager.cc:1918] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:34.827926 11663 log.cc:1196] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/f9e4559974db41e0ab3820ac16f0501a
I20260927 10:48:34.828303 11663 ts_tablet_manager.cc:1939] T f9e4559974db41e0ab3820ac16f0501a P 48f81b5ed713418485d4194140dab6ee: Deleting consensus metadata
I20260927 10:48:34.829368 11091 catalog_manager.cc:4841] TS 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747): tablet f9e4559974db41e0ab3820ac16f0501a (table test-table [id=07db5d3ccc6745f7be9467c87c0cd7b3]) successfully deleted
I20260927 10:48:34.829888 11383 tablet_service.cc:1467] Processing CreateTablet for tablet 8f32fded213d41268852a6b6446455ee (DEFAULT_TABLE table=test-table [id=6346316e601e4223868abc8a5e187ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:34.830118 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f32fded213d41268852a6b6446455ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.830114 11243 tablet_service.cc:1467] Processing CreateTablet for tablet 8f32fded213d41268852a6b6446455ee (DEFAULT_TABLE table=test-table [id=6346316e601e4223868abc8a5e187ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:34.830356 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f32fded213d41268852a6b6446455ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.832156 11613 tablet_bootstrap.cc:492] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Bootstrap starting.
I20260927 10:48:34.832482 11523 tablet_service.cc:1467] Processing CreateTablet for tablet 8f32fded213d41268852a6b6446455ee (DEFAULT_TABLE table=test-table [id=6346316e601e4223868abc8a5e187ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:34.832743 11523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f32fded213d41268852a6b6446455ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:34.832782 11614 tablet_bootstrap.cc:492] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Bootstrap starting.
I20260927 10:48:34.833508 11613 tablet_bootstrap.cc:654] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.833742 11614 tablet_bootstrap.cc:654] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.822790 11662 ts_tablet_manager.cc:1905] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:34.835501 11612 tablet_bootstrap.cc:492] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Bootstrap starting.
I20260927 10:48:34.836344 11612 tablet_bootstrap.cc:654] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:34.837682 11612 tablet_bootstrap.cc:492] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: No bootstrap required, opened a new log
I20260927 10:48:34.837762 11612 ts_tablet_manager.cc:1397] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:34.838207 11612 raft_consensus.cc:348] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.838320 11612 raft_consensus.cc:374] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.838374 11612 raft_consensus.cc:729] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9aa07b1b1da436199a6bea42eb10e88, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.838490 11612 consensus_queue.cc:260] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.838779 11612 ts_tablet_manager.cc:1428] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:34.840327 11662 ts_tablet_manager.cc:1918] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:34.840438 11664 tablet_replica.cc:330] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: stopping tablet replica
I20260927 10:48:34.840817 11664 raft_consensus.cc:2229] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:34.841362 11664 raft_consensus.cc:2258] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:34.841874 11614 tablet_bootstrap.cc:492] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: No bootstrap required, opened a new log
I20260927 10:48:34.842860 11614 ts_tablet_manager.cc:1397] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260927 10:48:34.843313 11614 raft_consensus.cc:348] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.843415 11614 raft_consensus.cc:374] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.843446 11614 raft_consensus.cc:729] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.843564 11614 consensus_queue.cc:260] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.843818 11614 ts_tablet_manager.cc:1428] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 10:48:34.844308 11662 log.cc:1196] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/f9e4559974db41e0ab3820ac16f0501a
I20260927 10:48:34.846899 11662 ts_tablet_manager.cc:1939] T f9e4559974db41e0ab3820ac16f0501a P cc31fe1a65594e738b80676f16ac3f57: Deleting consensus metadata
I20260927 10:48:34.847240 11664 ts_tablet_manager.cc:1905] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:34.847425 11613 tablet_bootstrap.cc:492] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: No bootstrap required, opened a new log
I20260927 10:48:34.847522 11613 ts_tablet_manager.cc:1397] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260927 10:48:34.847944 11613 raft_consensus.cc:348] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.848081 11613 raft_consensus.cc:374] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:34.848119 11613 raft_consensus.cc:729] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Initialized, Role: FOLLOWER
I20260927 10:48:34.848248 11613 consensus_queue.cc:260] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.848420 11090 catalog_manager.cc:4841] TS cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085): tablet f9e4559974db41e0ab3820ac16f0501a (table test-table [id=07db5d3ccc6745f7be9467c87c0cd7b3]) successfully deleted
I20260927 10:48:34.848556 11613 ts_tablet_manager.cc:1428] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:34.849961 11664 ts_tablet_manager.cc:1918] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:34.850044 11664 log.cc:1196] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/f9e4559974db41e0ab3820ac16f0501a
I20260927 10:48:34.850347 11664 ts_tablet_manager.cc:1939] T f9e4559974db41e0ab3820ac16f0501a P d9aa07b1b1da436199a6bea42eb10e88: Deleting consensus metadata
I20260927 10:48:34.851467 11090 catalog_manager.cc:4841] TS d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299): tablet f9e4559974db41e0ab3820ac16f0501a (table test-table [id=07db5d3ccc6745f7be9467c87c0cd7b3]) successfully deleted
I20260927 10:48:34.950023 11620 raft_consensus.cc:482] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:34.950176 11620 raft_consensus.cc:504] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.950625 11620 leader_election.cc:288] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.951073 11403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f32fded213d41268852a6b6446455ee" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f81b5ed713418485d4194140dab6ee" is_pre_election: true
I20260927 10:48:34.951064 11543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f32fded213d41268852a6b6446455ee" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88" is_pre_election: true
I20260927 10:48:34.951239 11543 raft_consensus.cc:2454] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 0.
I20260927 10:48:34.951242 11403 raft_consensus.cc:2454] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 0.
I20260927 10:48:34.951588 11194 leader_election.cc:302] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc31fe1a65594e738b80676f16ac3f57, d9aa07b1b1da436199a6bea42eb10e88; no voters:
I20260927 10:48:34.951776 11620 raft_consensus.cc:2781] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:34.951838 11620 raft_consensus.cc:482] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:34.951887 11620 raft_consensus.cc:3037] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.952842 11620 raft_consensus.cc:504] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.953181 11620 leader_election.cc:288] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 election: Requested vote from peers 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:34.953507 11403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f32fded213d41268852a6b6446455ee" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f81b5ed713418485d4194140dab6ee"
I20260927 10:48:34.953543 11543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f32fded213d41268852a6b6446455ee" candidate_uuid: "cc31fe1a65594e738b80676f16ac3f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88"
I20260927 10:48:34.953629 11403 raft_consensus.cc:3037] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.953650 11543 raft_consensus.cc:3037] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:34.954504 11403 raft_consensus.cc:2454] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 1.
I20260927 10:48:34.954726 11543 raft_consensus.cc:2454] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc31fe1a65594e738b80676f16ac3f57 in term 1.
I20260927 10:48:34.954921 11195 leader_election.cc:302] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, cc31fe1a65594e738b80676f16ac3f57; no voters:
I20260927 10:48:34.955082 11620 raft_consensus.cc:2781] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:34.955173 11620 raft_consensus.cc:686] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 1 LEADER]: Becoming Leader. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Running, Role: LEADER
I20260927 10:48:34.955314 11620 consensus_queue.cc:237] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:34.956871 11106 catalog_manager.cc:5495] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "cc31fe1a65594e738b80676f16ac3f57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:35.002465 11403 raft_consensus.cc:1261] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Refusing update from remote peer cc31fe1a65594e738b80676f16ac3f57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:35.002461 11543 raft_consensus.cc:1261] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Refusing update from remote peer cc31fe1a65594e738b80676f16ac3f57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:35.002883 11623 consensus_queue.cc:1035] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:35.003083 11620 consensus_queue.cc:1035] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 10:48:35.016808 11642 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 10:48:36.563304 11100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:36.563457 11100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:36.565587 11100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Sending DeleteTablet for 3 replicas of tablet 8f32fded213d41268852a6b6446455ee
I20260927 10:48:36.566079 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f32fded213d41268852a6b6446455ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:36 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40648
I20260927 10:48:36.566156 11523 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f32fded213d41268852a6b6446455ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:36 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:41554
I20260927 10:48:36.566179 11383 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f32fded213d41268852a6b6446455ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:36 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40672
I20260927 10:48:36.566591 11693 tablet_replica.cc:330] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: stopping tablet replica
I20260927 10:48:36.566776 11693 raft_consensus.cc:2229] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:36.566879 11693 raft_consensus.cc:2258] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:36.566411 11691 tablet_replica.cc:330] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: stopping tablet replica
I20260927 10:48:36.567236 11691 raft_consensus.cc:2229] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:36.567402 11693 ts_tablet_manager.cc:1905] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:36.567404 11691 raft_consensus.cc:2258] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:36.567613 11692 tablet_replica.cc:330] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: stopping tablet replica
I20260927 10:48:36.567754 11692 raft_consensus.cc:2229] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:36.567860 11692 raft_consensus.cc:2258] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:36.568254 11691 ts_tablet_manager.cc:1905] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:36.569561 11692 ts_tablet_manager.cc:1905] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:36.570385 11693 ts_tablet_manager.cc:1918] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 10:48:36.570459 11693 log.cc:1196] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/8f32fded213d41268852a6b6446455ee
I20260927 10:48:36.570760 11693 ts_tablet_manager.cc:1939] T 8f32fded213d41268852a6b6446455ee P 48f81b5ed713418485d4194140dab6ee: Deleting consensus metadata
I20260927 10:48:36.570837 11691 ts_tablet_manager.cc:1918] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 10:48:36.570919 11691 log.cc:1196] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/8f32fded213d41268852a6b6446455ee
I20260927 10:48:36.571245 11691 ts_tablet_manager.cc:1939] T 8f32fded213d41268852a6b6446455ee P cc31fe1a65594e738b80676f16ac3f57: Deleting consensus metadata
I20260927 10:48:36.571403 11091 catalog_manager.cc:4841] TS 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747): tablet 8f32fded213d41268852a6b6446455ee (table test-table [id=6346316e601e4223868abc8a5e187ec9]) successfully deleted
I20260927 10:48:36.572024 11692 ts_tablet_manager.cc:1918] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 10:48:36.572115 11692 log.cc:1196] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/8f32fded213d41268852a6b6446455ee
I20260927 10:48:36.572094 11090 catalog_manager.cc:4841] TS cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085): tablet 8f32fded213d41268852a6b6446455ee (table test-table [id=6346316e601e4223868abc8a5e187ec9]) successfully deleted
I20260927 10:48:36.572455 11692 ts_tablet_manager.cc:1939] T 8f32fded213d41268852a6b6446455ee P d9aa07b1b1da436199a6bea42eb10e88: Deleting consensus metadata
I20260927 10:48:36.573047 11090 catalog_manager.cc:4841] TS d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299): tablet 8f32fded213d41268852a6b6446455ee (table test-table [id=6346316e601e4223868abc8a5e187ec9]) successfully deleted
Sep 27 10:48:36 dist-test-slave-p9q6 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506116, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 10:48:36 dist-test-slave-p9q6 krb5kdc[11061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506116, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:36.600826 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:36.584806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.9.39.190:42861)
0927 10:48:36.585077 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:36.585086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:36.585107 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:48:36.585231 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:36.585234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:36.585275 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:36.585353 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:36.585981 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.586583 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.587459 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.587692 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.587801 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:36.596263 (+ 8462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:36.596268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:36.596271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:36.596301 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:36.599015 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:36.599519 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:36.599523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:36.599525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:36.599611 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:36.599925 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:36.599928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:36.599929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:36.600116 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:36.600245 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:36.600483 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 10:48:36.600589 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260927 10:48:36.603466 11100 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:36.603900 11100 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:36.608402 11243 tablet_service.cc:1467] Processing CreateTablet for tablet d85666a2c8354d12b20ace5ded60ae2a (DEFAULT_TABLE table=test-table [id=b783592e3b404221a67ee2435654503a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:36.608491 11383 tablet_service.cc:1467] Processing CreateTablet for tablet d85666a2c8354d12b20ace5ded60ae2a (DEFAULT_TABLE table=test-table [id=b783592e3b404221a67ee2435654503a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:36.608706 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d85666a2c8354d12b20ace5ded60ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:36.608814 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d85666a2c8354d12b20ace5ded60ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:36.608762 11523 tablet_service.cc:1467] Processing CreateTablet for tablet d85666a2c8354d12b20ace5ded60ae2a (DEFAULT_TABLE table=test-table [id=b783592e3b404221a67ee2435654503a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:36.609031 11523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d85666a2c8354d12b20ace5ded60ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:36.610834 11702 tablet_bootstrap.cc:492] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Bootstrap starting.
I20260927 10:48:36.611407 11703 tablet_bootstrap.cc:492] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Bootstrap starting.
I20260927 10:48:36.611773 11702 tablet_bootstrap.cc:654] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:36.611948 11704 tablet_bootstrap.cc:492] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Bootstrap starting.
I20260927 10:48:36.612623 11703 tablet_bootstrap.cc:654] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:36.613184 11702 tablet_bootstrap.cc:492] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: No bootstrap required, opened a new log
I20260927 10:48:36.613286 11702 ts_tablet_manager.cc:1397] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:36.613356 11704 tablet_bootstrap.cc:654] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:36.613793 11703 tablet_bootstrap.cc:492] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: No bootstrap required, opened a new log
I20260927 10:48:36.613749 11702 raft_consensus.cc:348] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.613868 11702 raft_consensus.cc:374] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:36.613905 11702 raft_consensus.cc:729] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Initialized, Role: FOLLOWER
I20260927 10:48:36.614024 11703 ts_tablet_manager.cc:1397] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:36.614046 11702 consensus_queue.cc:260] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.614346 11702 ts_tablet_manager.cc:1428] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 10:48:36.614642 11704 tablet_bootstrap.cc:492] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: No bootstrap required, opened a new log
I20260927 10:48:36.614784 11704 ts_tablet_manager.cc:1397] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:36.615091 11703 raft_consensus.cc:348] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.615180 11703 raft_consensus.cc:374] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:36.615234 11703 raft_consensus.cc:729] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Initialized, Role: FOLLOWER
I20260927 10:48:36.615374 11703 consensus_queue.cc:260] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.615365 11704 raft_consensus.cc:348] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.615540 11704 raft_consensus.cc:374] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:36.615585 11704 raft_consensus.cc:729] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9aa07b1b1da436199a6bea42eb10e88, State: Initialized, Role: FOLLOWER
I20260927 10:48:36.615746 11704 consensus_queue.cc:260] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.616180 11704 ts_tablet_manager.cc:1428] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:36.616732 11703 ts_tablet_manager.cc:1428] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260927 10:48:36.637421 11678 raft_consensus.cc:482] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:36.637559 11678 raft_consensus.cc:504] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.637928 11678 leader_election.cc:288] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:36.638296 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85666a2c8354d12b20ace5ded60ae2a" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc31fe1a65594e738b80676f16ac3f57" is_pre_election: true
I20260927 10:48:36.638439 11263 raft_consensus.cc:2454] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 0.
I20260927 10:48:36.638406 11542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85666a2c8354d12b20ace5ded60ae2a" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88" is_pre_election: true
I20260927 10:48:36.638568 11542 raft_consensus.cc:2454] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 0.
I20260927 10:48:36.638772 11334 leader_election.cc:302] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, cc31fe1a65594e738b80676f16ac3f57; no voters:
I20260927 10:48:36.638933 11678 raft_consensus.cc:2781] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:36.638988 11678 raft_consensus.cc:482] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:36.639027 11678 raft_consensus.cc:3037] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:36.639987 11678 raft_consensus.cc:504] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.640300 11678 leader_election.cc:288] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 election: Requested vote from peers cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:36.640614 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85666a2c8354d12b20ace5ded60ae2a" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc31fe1a65594e738b80676f16ac3f57"
I20260927 10:48:36.640666 11542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85666a2c8354d12b20ace5ded60ae2a" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88"
I20260927 10:48:36.640764 11263 raft_consensus.cc:3037] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:36.640820 11542 raft_consensus.cc:3037] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:36.641813 11263 raft_consensus.cc:2454] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 1.
I20260927 10:48:36.642004 11542 raft_consensus.cc:2454] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 1.
I20260927 10:48:36.642069 11334 leader_election.cc:302] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, cc31fe1a65594e738b80676f16ac3f57; no voters:
I20260927 10:48:36.642249 11678 raft_consensus.cc:2781] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:36.642360 11678 raft_consensus.cc:686] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 1 LEADER]: Becoming Leader. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Running, Role: LEADER
I20260927 10:48:36.642493 11678 consensus_queue.cc:237] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.643927 11105 catalog_manager.cc:5495] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 48f81b5ed713418485d4194140dab6ee (127.9.39.130). New cstate: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:36.660858 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:36.658003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52190 (local address 127.9.39.130:41747)
0927 10:48:36.658244 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:36.658250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:36.658281 (+ 31us) server_negotiation.cc:407] Connection header received
0927 10:48:36.658316 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:36.658318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:36.658354 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:36.658414 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:36.658625 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.659200 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.660005 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.660261 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.660336 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:36.660536 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:36.660599 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:36.660657 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:36.660710 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 10:48:36.660751 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":105,"threads_started":1}
I20260927 10:48:36.668793 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:36.666074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:51587 (local address 127.9.39.130:41747)
0927 10:48:36.666184 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:36.666188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:36.666324 (+ 136us) server_negotiation.cc:407] Connection header received
0927 10:48:36.666454 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:36.666456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:36.666493 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:36.666551 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:36.666824 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.667278 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.668012 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.668204 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.668272 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:36.668458 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:36.668510 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:36.668565 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:36.668635 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 10:48:36.668693 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":44}
I20260927 10:48:36.668807 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:36.666015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41747 (local address 127.9.39.130:51587)
0927 10:48:36.666249 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:36.666261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:36.666337 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:36.666580 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:36.666582 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:36.666589 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:48:36.666660 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:36.666664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.667363 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.667365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:36.667922 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:36.667929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:36.668292 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:36.668294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:36.668350 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:36.668371 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:48:36.668579 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:48:36.668583 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:48:36.668630 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 10:48:36.668701 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260927 10:48:36.673460 11542 raft_consensus.cc:1261] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Refusing update from remote peer 48f81b5ed713418485d4194140dab6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:36.673851 11636 consensus_queue.cc:1035] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:36.673942 11263 raft_consensus.cc:1261] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Refusing update from remote peer 48f81b5ed713418485d4194140dab6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:36.674374 11636 consensus_queue.cc:1035] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:36.677732 11674 mvcc.cc:205] Tried to move back new op lower bound from 7333913053891629056 to 7333913053768998912. Current Snapshot: MvccSnapshot[applied={T|T < 7333913053891629056}]
I20260927 10:48:36.706507 11105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:36.706686 11105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:36.708652 11105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Sending DeleteTablet for 3 replicas of tablet d85666a2c8354d12b20ace5ded60ae2a
I20260927 10:48:36.709256 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet d85666a2c8354d12b20ace5ded60ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:36 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40648
I20260927 10:48:36.709275 11382 tablet_service.cc:1514] Processing DeleteTablet for tablet d85666a2c8354d12b20ace5ded60ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:36 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40672
I20260927 10:48:36.709298 11523 tablet_service.cc:1514] Processing DeleteTablet for tablet d85666a2c8354d12b20ace5ded60ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:36 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:41554
I20260927 10:48:36.709434 11691 tablet_replica.cc:330] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: stopping tablet replica
I20260927 10:48:36.709440 11693 tablet_replica.cc:330] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: stopping tablet replica
I20260927 10:48:36.709444 11692 tablet_replica.cc:330] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: stopping tablet replica
I20260927 10:48:36.709534 11692 raft_consensus.cc:2229] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:36.709535 11691 raft_consensus.cc:2229] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:36.709556 11693 raft_consensus.cc:2229] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:36.709640 11692 raft_consensus.cc:2258] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:36.709648 11691 raft_consensus.cc:2258] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:36.709824 11693 raft_consensus.cc:2258] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:36.710394 11691 ts_tablet_manager.cc:1905] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:36.710855 11692 ts_tablet_manager.cc:1905] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:36.710937 11693 ts_tablet_manager.cc:1905] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:36.713115 11691 ts_tablet_manager.cc:1918] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:36.713186 11691 log.cc:1196] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/d85666a2c8354d12b20ace5ded60ae2a
I20260927 10:48:36.713307 11692 ts_tablet_manager.cc:1918] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:36.713371 11692 log.cc:1196] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/d85666a2c8354d12b20ace5ded60ae2a
I20260927 10:48:36.713383 11693 ts_tablet_manager.cc:1918] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:48:36.713446 11693 log.cc:1196] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/d85666a2c8354d12b20ace5ded60ae2a
I20260927 10:48:36.713447 11691 ts_tablet_manager.cc:1939] T d85666a2c8354d12b20ace5ded60ae2a P cc31fe1a65594e738b80676f16ac3f57: Deleting consensus metadata
I20260927 10:48:36.713645 11692 ts_tablet_manager.cc:1939] T d85666a2c8354d12b20ace5ded60ae2a P d9aa07b1b1da436199a6bea42eb10e88: Deleting consensus metadata
I20260927 10:48:36.713692 11693 ts_tablet_manager.cc:1939] T d85666a2c8354d12b20ace5ded60ae2a P 48f81b5ed713418485d4194140dab6ee: Deleting consensus metadata
I20260927 10:48:36.714274 11091 catalog_manager.cc:4841] TS 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747): tablet d85666a2c8354d12b20ace5ded60ae2a (table test-table [id=b783592e3b404221a67ee2435654503a]) successfully deleted
I20260927 10:48:36.714344 11090 catalog_manager.cc:4841] TS cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085): tablet d85666a2c8354d12b20ace5ded60ae2a (table test-table [id=b783592e3b404221a67ee2435654503a]) successfully deleted
I20260927 10:48:36.714553 11090 catalog_manager.cc:4841] TS d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299): tablet d85666a2c8354d12b20ace5ded60ae2a (table test-table [id=b783592e3b404221a67ee2435654503a]) successfully deleted
I20260927 10:48:36.714768 11105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:36.715162 11105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:36.719466 11243 tablet_service.cc:1467] Processing CreateTablet for tablet 9d67d89832ef41ffaeaef92d767dbf7c (DEFAULT_TABLE table=test-table [id=eb0f2649e72a4082824edb5877838eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:36.719501 11382 tablet_service.cc:1467] Processing CreateTablet for tablet 9d67d89832ef41ffaeaef92d767dbf7c (DEFAULT_TABLE table=test-table [id=eb0f2649e72a4082824edb5877838eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:36.719659 11523 tablet_service.cc:1467] Processing CreateTablet for tablet 9d67d89832ef41ffaeaef92d767dbf7c (DEFAULT_TABLE table=test-table [id=eb0f2649e72a4082824edb5877838eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:36.719717 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d67d89832ef41ffaeaef92d767dbf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:36.719756 11382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d67d89832ef41ffaeaef92d767dbf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:36.719890 11523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d67d89832ef41ffaeaef92d767dbf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:36.721804 11703 tablet_bootstrap.cc:492] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57: Bootstrap starting.
I20260927 10:48:36.721947 11702 tablet_bootstrap.cc:492] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Bootstrap starting.
I20260927 10:48:36.721947 11704 tablet_bootstrap.cc:492] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Bootstrap starting.
I20260927 10:48:36.722607 11703 tablet_bootstrap.cc:654] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:36.722823 11704 tablet_bootstrap.cc:654] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:36.722958 11702 tablet_bootstrap.cc:654] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:36.723663 11703 tablet_bootstrap.cc:492] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57: No bootstrap required, opened a new log
I20260927 10:48:36.723747 11703 ts_tablet_manager.cc:1397] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:36.724103 11703 raft_consensus.cc:348] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.724210 11703 raft_consensus.cc:374] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:36.724253 11702 tablet_bootstrap.cc:492] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: No bootstrap required, opened a new log
I20260927 10:48:36.724239 11703 raft_consensus.cc:729] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc31fe1a65594e738b80676f16ac3f57, State: Initialized, Role: FOLLOWER
I20260927 10:48:36.724328 11702 ts_tablet_manager.cc:1397] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 10:48:36.724363 11703 consensus_queue.cc:260] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.724606 11703 ts_tablet_manager.cc:1428] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:36.724747 11702 raft_consensus.cc:348] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.724853 11702 raft_consensus.cc:374] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:36.724895 11702 raft_consensus.cc:729] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Initialized, Role: FOLLOWER
I20260927 10:48:36.725018 11702 consensus_queue.cc:260] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.725272 11702 ts_tablet_manager.cc:1428] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 10:48:36.725284 11704 tablet_bootstrap.cc:492] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: No bootstrap required, opened a new log
I20260927 10:48:36.725351 11704 ts_tablet_manager.cc:1397] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260927 10:48:36.725791 11704 raft_consensus.cc:348] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.725907 11704 raft_consensus.cc:374] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:36.725950 11704 raft_consensus.cc:729] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9aa07b1b1da436199a6bea42eb10e88, State: Initialized, Role: FOLLOWER
I20260927 10:48:36.726079 11704 consensus_queue.cc:260] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.726322 11704 ts_tablet_manager.cc:1428] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 10:48:36.776928 11678 raft_consensus.cc:482] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:36.777093 11678 raft_consensus.cc:504] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.777475 11678 leader_election.cc:288] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:36.777920 11542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d67d89832ef41ffaeaef92d767dbf7c" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88" is_pre_election: true
I20260927 10:48:36.777938 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d67d89832ef41ffaeaef92d767dbf7c" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc31fe1a65594e738b80676f16ac3f57" is_pre_election: true
I20260927 10:48:36.778079 11542 raft_consensus.cc:2454] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 0.
I20260927 10:48:36.778090 11263 raft_consensus.cc:2454] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 0.
I20260927 10:48:36.778388 11334 leader_election.cc:302] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, cc31fe1a65594e738b80676f16ac3f57; no voters:
I20260927 10:48:36.778601 11678 raft_consensus.cc:2781] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:36.778669 11678 raft_consensus.cc:482] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:36.778748 11678 raft_consensus.cc:3037] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:36.779670 11678 raft_consensus.cc:504] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.779963 11678 leader_election.cc:288] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 election: Requested vote from peers cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085), d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299)
I20260927 10:48:36.780287 11542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d67d89832ef41ffaeaef92d767dbf7c" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9aa07b1b1da436199a6bea42eb10e88"
I20260927 10:48:36.780321 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d67d89832ef41ffaeaef92d767dbf7c" candidate_uuid: "48f81b5ed713418485d4194140dab6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc31fe1a65594e738b80676f16ac3f57"
I20260927 10:48:36.780407 11542 raft_consensus.cc:3037] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:36.780419 11263 raft_consensus.cc:3037] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:36.781255 11542 raft_consensus.cc:2454] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 1.
I20260927 10:48:36.781497 11263 raft_consensus.cc:2454] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f81b5ed713418485d4194140dab6ee in term 1.
I20260927 10:48:36.781531 11334 leader_election.cc:302] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f81b5ed713418485d4194140dab6ee, d9aa07b1b1da436199a6bea42eb10e88; no voters:
I20260927 10:48:36.781709 11678 raft_consensus.cc:2781] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:36.781791 11678 raft_consensus.cc:686] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 1 LEADER]: Becoming Leader. State: Replica: 48f81b5ed713418485d4194140dab6ee, State: Running, Role: LEADER
I20260927 10:48:36.781919 11678 consensus_queue.cc:237] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } }
I20260927 10:48:36.783514 11100 catalog_manager.cc:5495] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 48f81b5ed713418485d4194140dab6ee (127.9.39.130). New cstate: current_term: 1 leader_uuid: "48f81b5ed713418485d4194140dab6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48f81b5ed713418485d4194140dab6ee" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:36.815958 11263 raft_consensus.cc:1261] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Refusing update from remote peer 48f81b5ed713418485d4194140dab6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:36.816303 11542 raft_consensus.cc:1261] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Refusing update from remote peer 48f81b5ed713418485d4194140dab6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:36.816319 11722 consensus_queue.cc:1035] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc31fe1a65594e738b80676f16ac3f57" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:36.816676 11722 consensus_queue.cc:1035] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9aa07b1b1da436199a6bea42eb10e88" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 10:48:36.831054 11671 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 10:48:38.380781 11106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:38.380959 11106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:38.383076 11106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a7e85e21c994ef68e4496f56d12b4c9: Sending DeleteTablet for 3 replicas of tablet 9d67d89832ef41ffaeaef92d767dbf7c
I20260927 10:48:38.383518 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d67d89832ef41ffaeaef92d767dbf7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:38 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40648
I20260927 10:48:38.383575 11382 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d67d89832ef41ffaeaef92d767dbf7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:38 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40672
I20260927 10:48:38.383584 11523 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d67d89832ef41ffaeaef92d767dbf7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:38 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:41554
I20260927 10:48:38.383862 11744 tablet_replica.cc:330] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57: stopping tablet replica
I20260927 10:48:38.383929 11746 tablet_replica.cc:330] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: stopping tablet replica
I20260927 10:48:38.383960 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11178
I20260927 10:48:38.383999 11744 raft_consensus.cc:2229] T 9d67d89832ef41ffaeaef92d767dbf7c P cc31fe1a65594e738b80676f16ac3f57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:38.384634 11746 raft_consensus.cc:2229] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:38.384832 11746 raft_consensus.cc:2258] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:38.385237 11745 tablet_replica.cc:330] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: stopping tablet replica
I20260927 10:48:38.385371 11745 raft_consensus.cc:2229] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:38.385479 11745 raft_consensus.cc:2258] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:38.385985 11746 ts_tablet_manager.cc:1905] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:38.387094 11745 ts_tablet_manager.cc:1905] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:38.388512 11746 ts_tablet_manager.cc:1918] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 10:48:38.388592 11746 log.cc:1196] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/9d67d89832ef41ffaeaef92d767dbf7c
I20260927 10:48:38.388857 11746 ts_tablet_manager.cc:1939] T 9d67d89832ef41ffaeaef92d767dbf7c P 48f81b5ed713418485d4194140dab6ee: Deleting consensus metadata
I20260927 10:48:38.389541 11091 catalog_manager.cc:4841] TS 48f81b5ed713418485d4194140dab6ee (127.9.39.130:41747): tablet 9d67d89832ef41ffaeaef92d767dbf7c (table test-table [id=eb0f2649e72a4082824edb5877838eca]) successfully deleted
I20260927 10:48:38.389674 11745 ts_tablet_manager.cc:1918] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 10:48:38.389745 11745 log.cc:1196] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/9d67d89832ef41ffaeaef92d767dbf7c
I20260927 10:48:38.390057 11745 ts_tablet_manager.cc:1939] T 9d67d89832ef41ffaeaef92d767dbf7c P d9aa07b1b1da436199a6bea42eb10e88: Deleting consensus metadata
I20260927 10:48:38.390636 11090 catalog_manager.cc:4841] TS d9aa07b1b1da436199a6bea42eb10e88 (127.9.39.131:33299): tablet 9d67d89832ef41ffaeaef92d767dbf7c (table test-table [id=eb0f2649e72a4082824edb5877838eca]) successfully deleted
W20260927 10:48:38.391387 11090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:40085 (error 108)
I20260927 10:48:38.391685 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11318
I20260927 10:48:38.394196 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:38.393800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40085 (local address 127.0.0.1:35630)
0927 10:48:38.394058 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:38.394120 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":104,"threads_started":1}
W20260927 10:48:38.394322 11090 catalog_manager.cc:4584] TS cc31fe1a65594e738b80676f16ac3f57 (127.9.39.129:40085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d67d89832ef41ffaeaef92d767dbf7c: Network error: Client connection negotiation failed: client connection to 127.9.39.129:40085: connect: Connection refused (error 111)
I20260927 10:48:38.398904 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11458
I20260927 10:48:38.404656 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11074
2026-09-27T10:48:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6120 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:38 dist-test-slave-p9q6 krb5kdc[11751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:38 dist-test-slave-p9q6 krb5kdc[11751](info): set up 2 sockets
Sep 27 10:48:38 dist-test-slave-p9q6 krb5kdc[11751](info): commencing operation
krb5kdc: starting...
W20260927 10:48:39.442559 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:39 dist-test-slave-p9q6 krb5kdc[11751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:39.562151 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46449
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:39.671746 11764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:39.671988 11764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:39.672026 11764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:39.675978 11764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:39.676026 11764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:39.676045 11764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:39.676061 11764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:39.676076 11764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:39.681170 11764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:39.682148 11764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:39.683179 11764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:39.691265 11772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:39.691444 11764 server_base.cc:1034] running on GCE node
W20260927 10:48:39.691310 11770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:39.691277 11769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:39.692106 11764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:39.693136 11764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:39.694324 11764 hybrid_clock.cc:648] HybridClock initialized: now 1790506119694310 us; error 37 us; skew 500 ppm
Sep 27 10:48:39 dist-test-slave-p9q6 krb5kdc[11751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506119, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:39.696949 11764 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:39.697805 11764 webserver.cc:460] Webserver started at http://127.9.39.190:38455/ using document root <none> and password file <none>
I20260927 10:48:39.698364 11764 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:39.698472 11764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:39.698753 11764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:39.700730 11764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "9840ac56b705479c85a34875ef670c65"
format_stamp: "Formatted at 2026-09-27 10:48:39 on dist-test-slave-p9q6"
server_key: "772363fa58f0a6c5980f36b385870b2b"
server_key_iv: "f9b6385ac7d42b26de52ac82d11505d4"
server_key_version: "encryptionkey@0"
I20260927 10:48:39.701275 11764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "9840ac56b705479c85a34875ef670c65"
format_stamp: "Formatted at 2026-09-27 10:48:39 on dist-test-slave-p9q6"
server_key: "772363fa58f0a6c5980f36b385870b2b"
server_key_iv: "f9b6385ac7d42b26de52ac82d11505d4"
server_key_version: "encryptionkey@0"
I20260927 10:48:39.704841 11764 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260927 10:48:39.707113 11779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:39.708288 11764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:39.708429 11764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "9840ac56b705479c85a34875ef670c65"
format_stamp: "Formatted at 2026-09-27 10:48:39 on dist-test-slave-p9q6"
server_key: "772363fa58f0a6c5980f36b385870b2b"
server_key_iv: "f9b6385ac7d42b26de52ac82d11505d4"
server_key_version: "encryptionkey@0"
I20260927 10:48:39.708535 11764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:39.722859 11764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:39.725411 11764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:39.725620 11764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:39.732399 11764 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:46449
I20260927 10:48:39.732429 11831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:46449 every 8 connection(s)
I20260927 10:48:39.733327 11764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:39.735904 11832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:39.737771 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11764
I20260927 10:48:39.737885 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:39.738153 9374 external_mini_cluster.cc:1428] Setting key 5d0949d072da8cefb2251c99afad2101
I20260927 10:48:39.741983 11832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65: Bootstrap starting.
I20260927 10:48:39.744331 11832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:39.745117 11832 log.cc:824] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65: Log is configured to *not* fsync() on all Append() calls
Sep 27 10:48:39 dist-test-slave-p9q6 krb5kdc[11751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:39.747174 11832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65: No bootstrap required, opened a new log
I20260927 10:48:39.750542 11832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9840ac56b705479c85a34875ef670c65" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46449 } }
I20260927 10:48:39.750793 11832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:39.750851 11832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9840ac56b705479c85a34875ef670c65, State: Initialized, Role: FOLLOWER
I20260927 10:48:39.751289 11832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9840ac56b705479c85a34875ef670c65" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46449 } }
I20260927 10:48:39.751441 11832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:39.751508 11832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:39.751605 11832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:39.752391 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:39.739509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.9.39.190:46449)
0927 10:48:39.739895 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:39.739908 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:39.739947 (+ 39us) server_negotiation.cc:407] Connection header received
0927 10:48:39.740553 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:39.740570 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:39.740861 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:39.741199 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:39.741961 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:39.743023 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:39.743964 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:39.744497 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:39.744666 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:39.747481 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:39.747486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:39.747498 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:39.747523 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:39.749611 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:39.750046 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:39.750052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:39.750057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:39.750167 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:39.750446 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:39.750450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:39.750452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:39.750924 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:39.751124 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:39.751348 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 10:48:39.751729 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":127,"threads_started":1}
I20260927 10:48:39.752413 11832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9840ac56b705479c85a34875ef670c65" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46449 } }
I20260927 10:48:39.752781 11832 leader_election.cc:302] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9840ac56b705479c85a34875ef670c65; no voters:
I20260927 10:48:39.753115 11832 leader_election.cc:288] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:39.753151 11837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:39.753311 11837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [term 1 LEADER]: Becoming Leader. State: Replica: 9840ac56b705479c85a34875ef670c65, State: Running, Role: LEADER
I20260927 10:48:39.753638 11837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9840ac56b705479c85a34875ef670c65" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46449 } }
I20260927 10:48:39.754251 11832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:39.754810 11838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9840ac56b705479c85a34875ef670c65. Latest consensus state: current_term: 1 leader_uuid: "9840ac56b705479c85a34875ef670c65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9840ac56b705479c85a34875ef670c65" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46449 } } }
I20260927 10:48:39.754918 11838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:39.754858 11839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9840ac56b705479c85a34875ef670c65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9840ac56b705479c85a34875ef670c65" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46449 } } }
I20260927 10:48:39.754983 11839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:39.755257 11842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:39.758287 11842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:39.763800 11842 catalog_manager.cc:1347] Generated new cluster ID: 6d123f0682024ec8be43c7b5ac13ea91
I20260927 10:48:39.763860 11842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:39.771332 11842 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:39.772455 11842 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:39.777194 11842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9840ac56b705479c85a34875ef670c65: Generated new TSK 0
I20260927 10:48:39.777873 11842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:39.805938 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46449
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:39.913259 11858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:39.913537 11858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:39.913610 11858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:39.917629 11858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:39.917723 11858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:39.917816 11858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:39.923089 11858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:39.924136 11858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:39.925160 11858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:39.934543 11864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:39.934578 11866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:39.934772 11858 server_base.cc:1034] running on GCE node
W20260927 10:48:39.934962 11863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:39.935344 11858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:39.935954 11858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:39.937136 11858 hybrid_clock.cc:648] HybridClock initialized: now 1790506119937119 us; error 28 us; skew 500 ppm
Sep 27 10:48:39 dist-test-slave-p9q6 krb5kdc[11751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506119, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:39.939769 11858 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:39.940572 11858 webserver.cc:460] Webserver started at http://127.9.39.129:44819/ using document root <none> and password file <none>
I20260927 10:48:39.941111 11858 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:39.941226 11858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:39.941447 11858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:39.943509 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "c2377b55b756446fbe5632840bcf3c6e"
format_stamp: "Formatted at 2026-09-27 10:48:39 on dist-test-slave-p9q6"
server_key: "860ab0cd284f0bc80be8b6cc69fb7b6c"
server_key_iv: "5efe607c67f3ac313e7592e33a7bbceb"
server_key_version: "encryptionkey@0"
I20260927 10:48:39.944119 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "c2377b55b756446fbe5632840bcf3c6e"
format_stamp: "Formatted at 2026-09-27 10:48:39 on dist-test-slave-p9q6"
server_key: "860ab0cd284f0bc80be8b6cc69fb7b6c"
server_key_iv: "5efe607c67f3ac313e7592e33a7bbceb"
server_key_version: "encryptionkey@0"
I20260927 10:48:39.947670 11858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:39.949864 11873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:39.951076 11858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:39.951205 11858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "c2377b55b756446fbe5632840bcf3c6e"
format_stamp: "Formatted at 2026-09-27 10:48:39 on dist-test-slave-p9q6"
server_key: "860ab0cd284f0bc80be8b6cc69fb7b6c"
server_key_iv: "5efe607c67f3ac313e7592e33a7bbceb"
server_key_version: "encryptionkey@0"
I20260927 10:48:39.951306 11858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:39.983704 11858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:39.986533 11858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:39.986796 11858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:39.987378 11858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:39.988185 11858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:39.988255 11858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:39.988327 11858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:39.988377 11858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:39.997042 11858 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:35041
I20260927 10:48:39.997058 11986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:35041 every 8 connection(s)
I20260927 10:48:39.998050 11858 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:40.002349 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11858
I20260927 10:48:40.002447 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:40.002755 9374 external_mini_cluster.cc:1428] Setting key ac209ae7026521e221c29ce643d15146
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[11751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506119, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:40.012544 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:39.999759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:33399 (local address 127.9.39.190:46449)
0927 10:48:39.999904 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:39.999909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:40.000731 (+ 822us) server_negotiation.cc:407] Connection header received
0927 10:48:40.001941 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:40.001943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:40.001987 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:40.002058 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:40.003485 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.004090 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.005210 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.005649 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.005751 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:40.008550 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:40.008558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:40.008561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:40.008593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:40.010318 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:40.010839 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:40.010845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:40.010848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:40.010943 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:40.011296 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:40.011302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:40.011306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:40.011561 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:40.011660 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:40.012109 (+ 449us) server_negotiation.cc:299] Negotiation successful
0927 10:48:40.012282 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:48:40.013254 11989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:40.000164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46449 (local address 127.9.39.129:33399)
0927 10:48:40.000587 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:40.000629 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:40.001747 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:40.002220 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:40.002227 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:40.002640 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:40.003351 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:40.003368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.004234 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.004237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:40.005090 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:40.005099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.005826 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.005832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:40.006054 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:40.006676 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:40.006718 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:40.008375 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:40.010438 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:40.010443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:40.010454 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:40.010738 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:40.011042 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:40.011044 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:40.011046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:40.011212 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:40.011662 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:40.011667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:40.011868 (+ 201us) client_negotiation.cc:769] Sending connection context
0927 10:48:40.012010 (+ 142us) client_negotiation.cc:240] Negotiation successful
0927 10:48:40.012336 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260927 10:48:40.014456 11987 heartbeater.cc:347] Connected to a master server at 127.9.39.190:46449
I20260927 10:48:40.014684 11987 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:48:40.015242 11987 heartbeater.cc:511] Master 127.9.39.190:46449 requested a full tablet report, sending...
I20260927 10:48:40.016742 11796 ts_manager.cc:194] Registered new tserver with Master: c2377b55b756446fbe5632840bcf3c6e (127.9.39.129:35041)
I20260927 10:48:40.018392 11796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:33399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:40.030172 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46449
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:40.139848 11992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:40.140074 11992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:40.140153 11992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:40.144184 11992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:40.144240 11992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:40.144364 11992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:40.149515 11992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:40.150496 11992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:40.151537 11992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:40.161085 11997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:40.161181 12000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:40.161085 11998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:40.161346 11992 server_base.cc:1034] running on GCE node
I20260927 10:48:40.161823 11992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:40.162451 11992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:40.163604 11992 hybrid_clock.cc:648] HybridClock initialized: now 1790506120163580 us; error 38 us; skew 500 ppm
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[11751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506120, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:40.166467 11992 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:40.167289 11992 webserver.cc:460] Webserver started at http://127.9.39.130:33105/ using document root <none> and password file <none>
I20260927 10:48:40.167765 11992 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:40.167819 11992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:40.167982 11992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:40.169866 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "21a6c9327f7f44dc96e1f81a92b0b8bb"
format_stamp: "Formatted at 2026-09-27 10:48:40 on dist-test-slave-p9q6"
server_key: "61289dacb8dcf75d691f9d70d6bed23c"
server_key_iv: "1fdbc84d372d07e0fa6fdfb0443cfa6b"
server_key_version: "encryptionkey@0"
I20260927 10:48:40.170337 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "21a6c9327f7f44dc96e1f81a92b0b8bb"
format_stamp: "Formatted at 2026-09-27 10:48:40 on dist-test-slave-p9q6"
server_key: "61289dacb8dcf75d691f9d70d6bed23c"
server_key_iv: "1fdbc84d372d07e0fa6fdfb0443cfa6b"
server_key_version: "encryptionkey@0"
I20260927 10:48:40.173744 11992 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:48:40.176177 12007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:40.177356 11992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:40.177472 11992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "21a6c9327f7f44dc96e1f81a92b0b8bb"
format_stamp: "Formatted at 2026-09-27 10:48:40 on dist-test-slave-p9q6"
server_key: "61289dacb8dcf75d691f9d70d6bed23c"
server_key_iv: "1fdbc84d372d07e0fa6fdfb0443cfa6b"
server_key_version: "encryptionkey@0"
I20260927 10:48:40.177577 11992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:40.189685 11992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:40.198827 11992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:40.198997 11992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:40.199635 11992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:40.200495 11992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:40.200553 11992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:40.200639 11992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:40.200721 11992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:40.210060 11992 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:33197
I20260927 10:48:40.210103 12120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:33197 every 8 connection(s)
I20260927 10:48:40.211055 11992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:40.216177 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 11992
I20260927 10:48:40.216274 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:40.216572 9374 external_mini_cluster.cc:1428] Setting key 4b02b78692f6dd774335b75afc94f816
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[11751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506120, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:40.225175 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:40.212891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:45737 (local address 127.9.39.190:46449)
0927 10:48:40.213018 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:40.213022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:40.213852 (+ 830us) server_negotiation.cc:407] Connection header received
0927 10:48:40.215097 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:40.215101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:40.215155 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:40.215230 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:40.216375 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.216997 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.218274 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.218489 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.218567 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:40.221323 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:40.221330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:40.221336 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:40.221368 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:40.223012 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:40.223572 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:40.223577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:40.223579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:40.223666 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:40.224036 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:40.224039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:40.224041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:40.224231 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:40.224370 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:40.224866 (+ 496us) server_negotiation.cc:299] Negotiation successful
0927 10:48:40.224994 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:48:40.225977 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:40.213306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46449 (local address 127.9.39.130:45737)
0927 10:48:40.213710 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:40.213748 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:40.214891 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:40.215414 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:40.215421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:40.215758 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:40.216210 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:40.216223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.217132 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.217141 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:40.218150 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:40.218162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.218650 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.218657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:40.218934 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:40.219563 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:40.219591 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:40.221152 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:40.223151 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:40.223159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:40.223173 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:40.223462 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:40.223773 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:40.223777 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:40.223779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:40.223926 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:40.224380 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:40.224389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:40.224616 (+ 227us) client_negotiation.cc:769] Sending connection context
0927 10:48:40.224817 (+ 201us) client_negotiation.cc:240] Negotiation successful
0927 10:48:40.225130 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":117,"threads_started":1}
I20260927 10:48:40.227296 12121 heartbeater.cc:347] Connected to a master server at 127.9.39.190:46449
I20260927 10:48:40.227592 12121 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:48:40.228216 12121 heartbeater.cc:511] Master 127.9.39.190:46449 requested a full tablet report, sending...
I20260927 10:48:40.229275 11796 ts_manager.cc:194] Registered new tserver with Master: 21a6c9327f7f44dc96e1f81a92b0b8bb (127.9.39.130:33197)
I20260927 10:48:40.229833 11796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:45737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:40.243844 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46449
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:40.354905 12126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:40.355242 12126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:40.355340 12126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:40.362471 12126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:40.362550 12126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:40.362643 12126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:40.369683 12126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:40.370810 12126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:40.372166 12126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:48:40.384640 12126 server_base.cc:1034] running on GCE node
W20260927 10:48:40.384662 12131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:40.384797 12134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:40.384933 12132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:40.385327 12126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:40.385983 12126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:40.387166 12126 hybrid_clock.cc:648] HybridClock initialized: now 1790506120387137 us; error 46 us; skew 500 ppm
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[11751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506120, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:40.390014 12126 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:40.390954 12126 webserver.cc:460] Webserver started at http://127.9.39.131:38141/ using document root <none> and password file <none>
I20260927 10:48:40.391486 12126 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:40.391546 12126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:40.391773 12126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:40.393671 12126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "ed9e71587d004f80936495b464b1dfa7"
format_stamp: "Formatted at 2026-09-27 10:48:40 on dist-test-slave-p9q6"
server_key: "7e5991e74844637abf30739d9e89608a"
server_key_iv: "97485d41699e07610a0630eea4f8bbca"
server_key_version: "encryptionkey@0"
I20260927 10:48:40.394213 12126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "ed9e71587d004f80936495b464b1dfa7"
format_stamp: "Formatted at 2026-09-27 10:48:40 on dist-test-slave-p9q6"
server_key: "7e5991e74844637abf30739d9e89608a"
server_key_iv: "97485d41699e07610a0630eea4f8bbca"
server_key_version: "encryptionkey@0"
I20260927 10:48:40.397786 12126 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:48:40.400111 12141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:40.401253 12126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:40.401382 12126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "ed9e71587d004f80936495b464b1dfa7"
format_stamp: "Formatted at 2026-09-27 10:48:40 on dist-test-slave-p9q6"
server_key: "7e5991e74844637abf30739d9e89608a"
server_key_iv: "97485d41699e07610a0630eea4f8bbca"
server_key_version: "encryptionkey@0"
I20260927 10:48:40.401499 12126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:40.423032 12126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:40.425729 12126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:40.425928 12126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:40.426505 12126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:40.427363 12126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:40.427443 12126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:40.427537 12126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:40.427590 12126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:40.436496 12126 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:41529
I20260927 10:48:40.436517 12254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:41529 every 8 connection(s)
I20260927 10:48:40.437345 12126 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:40.439823 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12126
I20260927 10:48:40.439953 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:40.440251 9374 external_mini_cluster.cc:1428] Setting key 5473bbcd626e4950951a59b7b4a34aa0
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[11751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506120, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:40.450656 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:40.439063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:36989 (local address 127.9.39.190:46449)
0927 10:48:40.439187 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:40.439191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:40.440018 (+ 827us) server_negotiation.cc:407] Connection header received
0927 10:48:40.441627 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:40.441630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:40.441683 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:40.441780 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:40.443134 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.443654 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.444760 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.445058 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.445176 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:40.447489 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:40.447493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:40.447496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:40.447528 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:40.448726 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:40.449207 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:40.449212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:40.449214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:40.449295 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:40.449622 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:40.449627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:40.449628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:40.449802 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:40.449922 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:40.450383 (+ 461us) server_negotiation.cc:299] Negotiation successful
0927 10:48:40.450494 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:48:40.451483 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:40.439362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46449 (local address 127.9.39.131:36989)
0927 10:48:40.439852 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:40.439912 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:40.441414 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:40.441948 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:40.441954 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:40.442470 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:40.442981 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:40.442995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.443772 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.443775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:40.444638 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:40.444646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:40.445208 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:40.445212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:40.445382 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:40.445944 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:40.445960 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:40.447324 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:40.448839 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:40.448845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:40.448857 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:40.449106 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:40.449395 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:40.449398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:40.449400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:40.449528 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:40.449922 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:40.449929 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:40.450155 (+ 226us) client_negotiation.cc:769] Sending connection context
0927 10:48:40.450330 (+ 175us) client_negotiation.cc:240] Negotiation successful
0927 10:48:40.450631 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":111,"threads_started":1}
I20260927 10:48:40.452551 12255 heartbeater.cc:347] Connected to a master server at 127.9.39.190:46449
I20260927 10:48:40.452793 12255 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:40.453316 12255 heartbeater.cc:511] Master 127.9.39.190:46449 requested a full tablet report, sending...
I20260927 10:48:40.454327 11796 ts_manager.cc:194] Registered new tserver with Master: ed9e71587d004f80936495b464b1dfa7 (127.9.39.131:41529)
I20260927 10:48:40.455063 11796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:36989
I20260927 10:48:40.465641 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 10:48:40.477923 11835 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:40.476406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.9.39.190:46449)
0927 10:48:40.476553 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:40.476559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:40.476737 (+ 178us) server_negotiation.cc:407] Connection header received
0927 10:48:40.476917 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:40.476919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:40.476960 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:40.477045 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:40.477741 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47126: BlockingRecv error: recv got EOF from 127.0.0.1:47126 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 10:48:40.478890 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11858
I20260927 10:48:40.484169 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11992
I20260927 10:48:40.489332 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12126
I20260927 10:48:40.494266 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 11764
2026-09-27T10:48:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2088 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[12267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[12267](info): set up 2 sockets
Sep 27 10:48:40 dist-test-slave-p9q6 krb5kdc[12267](info): commencing operation
krb5kdc: starting...
W20260927 10:48:41.530578 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:41 dist-test-slave-p9q6 krb5kdc[12267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:41.651816 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46325
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:35109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:41.760921 12280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:41.761299 12280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:41.761358 12280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:41.765921 12280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:41.765981 12280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:41.766001 12280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:41.766018 12280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:41.766033 12280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:41.771445 12280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:35109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:41.772398 12280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:41.773355 12280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:48:41.781598 12280 server_base.cc:1034] running on GCE node
W20260927 10:48:41.781558 12286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:41.781582 12285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:41.781558 12288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:41.782327 12280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:41.783421 12280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:41.784576 12280 hybrid_clock.cc:648] HybridClock initialized: now 1790506121784547 us; error 47 us; skew 500 ppm
Sep 27 10:48:41 dist-test-slave-p9q6 krb5kdc[12267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:41.792379 12280 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:41.793279 12280 webserver.cc:460] Webserver started at http://127.9.39.190:36847/ using document root <none> and password file <none>
I20260927 10:48:41.793883 12280 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:41.793946 12280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:41.794173 12280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:41.796240 12280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "7eba9478ad844d30abff9d0cc3015c6f"
format_stamp: "Formatted at 2026-09-27 10:48:41 on dist-test-slave-p9q6"
server_key: "ade164684b56dc72ae14d20010573acb"
server_key_iv: "70d0a6335280c4957b8ff2928a7b4075"
server_key_version: "encryptionkey@0"
I20260927 10:48:41.796799 12280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "7eba9478ad844d30abff9d0cc3015c6f"
format_stamp: "Formatted at 2026-09-27 10:48:41 on dist-test-slave-p9q6"
server_key: "ade164684b56dc72ae14d20010573acb"
server_key_iv: "70d0a6335280c4957b8ff2928a7b4075"
server_key_version: "encryptionkey@0"
I20260927 10:48:41.800405 12280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:48:41.802665 12295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:41.803874 12280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:41.804008 12280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "7eba9478ad844d30abff9d0cc3015c6f"
format_stamp: "Formatted at 2026-09-27 10:48:41 on dist-test-slave-p9q6"
server_key: "ade164684b56dc72ae14d20010573acb"
server_key_iv: "70d0a6335280c4957b8ff2928a7b4075"
server_key_version: "encryptionkey@0"
I20260927 10:48:41.804116 12280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:41.815828 12280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:41.818673 12280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:41.818935 12280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:41.825702 12280 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:46325
I20260927 10:48:41.825991 12347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:46325 every 8 connection(s)
I20260927 10:48:41.826637 12280 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:41.827471 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12280
I20260927 10:48:41.827605 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:41.827858 9374 external_mini_cluster.cc:1428] Setting key 87cb4e42617cf658843ef82a3a7d10e1
I20260927 10:48:41.829815 12348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:41.836226 12348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f: Bootstrap starting.
Sep 27 10:48:41 dist-test-slave-p9q6 krb5kdc[12267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:41.838783 12348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:41.839637 12348 log.cc:824] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:41.841598 12348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f: No bootstrap required, opened a new log
I20260927 10:48:41.842339 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:41.829159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.9.39.190:46325)
0927 10:48:41.829588 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:41.829605 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:41.829647 (+ 42us) server_negotiation.cc:407] Connection header received
0927 10:48:41.830450 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:41.830471 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:41.830896 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:41.831244 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:41.831881 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:41.833366 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:41.834183 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:41.834906 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:41.835072 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:41.837494 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:41.837500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:41.837515 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:41.837543 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:41.839499 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:41.839972 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:41.839979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:41.839986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:41.840124 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:41.840429 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:41.840432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:41.840434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:41.840812 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:41.841011 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:41.841316 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 10:48:41.841589 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":113,"threads_started":1}
I20260927 10:48:41.845196 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:48:41.845604 12348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eba9478ad844d30abff9d0cc3015c6f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46325 } }
I20260927 10:48:41.845839 12348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:41.845927 12348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eba9478ad844d30abff9d0cc3015c6f, State: Initialized, Role: FOLLOWER
I20260927 10:48:41.846333 12348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eba9478ad844d30abff9d0cc3015c6f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46325 } }
I20260927 10:48:41.846498 12348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:41.846580 12348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:41.846717 12348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:41.847589 12348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eba9478ad844d30abff9d0cc3015c6f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46325 } }
I20260927 10:48:41.847921 12348 leader_election.cc:302] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eba9478ad844d30abff9d0cc3015c6f; no voters:
I20260927 10:48:41.848410 12348 leader_election.cc:288] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:41.848508 12354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:41.848737 12354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [term 1 LEADER]: Becoming Leader. State: Replica: 7eba9478ad844d30abff9d0cc3015c6f, State: Running, Role: LEADER
I20260927 10:48:41.849056 12354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eba9478ad844d30abff9d0cc3015c6f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46325 } }
I20260927 10:48:41.849586 12348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:41.850673 12355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eba9478ad844d30abff9d0cc3015c6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eba9478ad844d30abff9d0cc3015c6f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46325 } } }
I20260927 10:48:41.850823 12355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:41.851042 12355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eba9478ad844d30abff9d0cc3015c6f. Latest consensus state: current_term: 1 leader_uuid: "7eba9478ad844d30abff9d0cc3015c6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eba9478ad844d30abff9d0cc3015c6f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46325 } } }
I20260927 10:48:41.851162 12355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:41.851444 12363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:41.854450 12363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:41.859889 12363 catalog_manager.cc:1347] Generated new cluster ID: bf716b277fb34aae9c013eb1ff89e901
I20260927 10:48:41.859958 12363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:41.864913 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:41.860301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60866 (local address 127.9.39.190:46325)
0927 10:48:41.860437 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:41.860442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:41.860688 (+ 246us) server_negotiation.cc:407] Connection header received
0927 10:48:41.860847 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:41.860851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:41.860900 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:41.860990 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:41.861479 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:41.862024 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:41.862919 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:41.863160 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:41.863270 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:41.864095 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:41.864099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:41.864102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:41.864499 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:41.864525 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:41.864540 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:41.864637 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:41.864716 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 10:48:41.864770 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 10:48:41.880508 12363 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:41.881392 12363 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:41.888047 12363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eba9478ad844d30abff9d0cc3015c6f: Generated new TSK 0
I20260927 10:48:41.888792 12363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:48:41.900516 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12280
2026-09-27T10:48:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1403 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:41 dist-test-slave-p9q6 krb5kdc[12381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:41 dist-test-slave-p9q6 krb5kdc[12381](info): set up 2 sockets
Sep 27 10:48:41 dist-test-slave-p9q6 krb5kdc[12381](info): commencing operation
krb5kdc: starting...
W20260927 10:48:42.934660 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:43.053359 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:34303
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:43.160141 12394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:43.160410 12394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:43.160487 12394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:43.164470 12394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:43.164546 12394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:43.164582 12394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:43.164655 12394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:43.164705 12394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:43.169898 12394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:43.170961 12394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:43.171962 12394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:43.179943 12399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.180037 12394 server_base.cc:1034] running on GCE node
W20260927 10:48:43.180040 12400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:43.179945 12402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.180722 12394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:43.181716 12394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:43.182871 12394 hybrid_clock.cc:648] HybridClock initialized: now 1790506123182842 us; error 39 us; skew 500 ppm
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:43.185475 12394 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:43.186244 12394 webserver.cc:460] Webserver started at http://127.9.39.190:36583/ using document root <none> and password file <none>
I20260927 10:48:43.186825 12394 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:43.186911 12394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:43.187117 12394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:43.189052 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "3921b3fa93aa5eb77f7e7014ed4e0a8e"
server_key_iv: "789816b39e9064a3e1b5993c0dc89d0c"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.189625 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "3921b3fa93aa5eb77f7e7014ed4e0a8e"
server_key_iv: "789816b39e9064a3e1b5993c0dc89d0c"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.193138 12394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 10:48:43.195513 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.196708 12394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:48:43.196837 12394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "3921b3fa93aa5eb77f7e7014ed4e0a8e"
server_key_iv: "789816b39e9064a3e1b5993c0dc89d0c"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.196938 12394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:43.218799 12394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:43.221906 12394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:43.222095 12394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:43.228521 12394 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:34303
I20260927 10:48:43.228834 12461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:34303 every 8 connection(s)
I20260927 10:48:43.229432 12394 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:43.232089 12462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:43.237093 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8: Bootstrap starting.
I20260927 10:48:43.238839 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12394
I20260927 10:48:43.238976 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:43.239223 9374 external_mini_cluster.cc:1428] Setting key 130b99d0b980749d55545a3ec76420a4
I20260927 10:48:43.239442 12462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:43.240283 12462 log.cc:824] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:43.242020 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8: No bootstrap required, opened a new log
I20260927 10:48:43.245277 12462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34303 } }
I20260927 10:48:43.245519 12462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:43.245592 12462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6e54b36f7d48cc8f2a9f7d603164f8, State: Initialized, Role: FOLLOWER
I20260927 10:48:43.246068 12462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34303 } }
I20260927 10:48:43.246214 12462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:43.246300 12462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:43.246397 12462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:43.247495 12462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34303 } }
I20260927 10:48:43.247851 12462 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6e54b36f7d48cc8f2a9f7d603164f8; no voters:
I20260927 10:48:43.248222 12462 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:43.248324 12467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:43.248835 12467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [term 1 LEADER]: Becoming Leader. State: Replica: 8f6e54b36f7d48cc8f2a9f7d603164f8, State: Running, Role: LEADER
I20260927 10:48:43.249147 12467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34303 } }
I20260927 10:48:43.249434 12462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:43.250742 12468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34303 } } }
I20260927 10:48:43.250895 12468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:43.251096 12468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6e54b36f7d48cc8f2a9f7d603164f8. Latest consensus state: current_term: 1 leader_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e54b36f7d48cc8f2a9f7d603164f8" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34303 } } }
I20260927 10:48:43.251178 12468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:43.253537 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.240252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.9.39.190:34303)
0927 10:48:43.240699 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:43.240717 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:43.240758 (+ 41us) server_negotiation.cc:407] Connection header received
0927 10:48:43.241440 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:43.241455 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:43.241747 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:43.242062 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:43.242876 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.243791 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.244634 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.245109 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.245279 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.248151 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:43.248156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:43.248167 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:43.248208 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:43.250261 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.250906 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.250914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.250919 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.251058 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.251395 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.251398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.251399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.251988 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:43.252189 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:43.252433 (+ 244us) server_negotiation.cc:299] Negotiation successful
0927 10:48:43.252711 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
W20260927 10:48:43.254195 12482 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:48:43.254300 12482 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:48:43.254506 12478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:43.255314 12478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:43.260738 12478 catalog_manager.cc:1347] Generated new cluster ID: 705c2643121b4932a88a1a43cea49d42
I20260927 10:48:43.260830 12478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:43.277057 12478 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:43.277909 12478 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:43.283347 12478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6e54b36f7d48cc8f2a9f7d603164f8: Generated new TSK 0
I20260927 10:48:43.283977 12478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:43.319883 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34303
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:43.435932 12488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:43.436201 12488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:43.436275 12488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:43.440205 12488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:43.440290 12488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:43.440375 12488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:43.445537 12488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:43.446540 12488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:43.447572 12488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:43.457231 12494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:43.457255 12493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.457403 12488 server_base.cc:1034] running on GCE node
W20260927 10:48:43.457264 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.457787 12488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:43.458431 12488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:43.459614 12488 hybrid_clock.cc:648] HybridClock initialized: now 1790506123459581 us; error 43 us; skew 500 ppm
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:43.462309 12488 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:43.463269 12488 webserver.cc:460] Webserver started at http://127.9.39.129:37837/ using document root <none> and password file <none>
I20260927 10:48:43.463855 12488 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:43.463954 12488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:43.464203 12488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:43.466809 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "c48410cc68a546a08c1b7c7650277c79"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "1b86b535dd5c423a55dd23f4243136bb"
server_key_iv: "de6f1087f83de7451e34bed4533ca8c4"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.467422 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "c48410cc68a546a08c1b7c7650277c79"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "1b86b535dd5c423a55dd23f4243136bb"
server_key_iv: "de6f1087f83de7451e34bed4533ca8c4"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.471309 12488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 10:48:43.473755 12503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.475070 12488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:43.475206 12488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "c48410cc68a546a08c1b7c7650277c79"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "1b86b535dd5c423a55dd23f4243136bb"
server_key_iv: "de6f1087f83de7451e34bed4533ca8c4"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.475327 12488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:43.489657 12488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:43.492472 12488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:43.492677 12488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:43.493233 12488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:43.494022 12488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:43.494093 12488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.494163 12488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:43.494205 12488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.502772 12488 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:35975
I20260927 10:48:43.502823 12616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:35975 every 8 connection(s)
I20260927 10:48:43.503657 12488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:43.506196 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12488
I20260927 10:48:43.506314 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:43.506629 9374 external_mini_cluster.cc:1428] Setting key 31ac9f1ff77668107ff709de0e1b1c91
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:48:43.518471 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.505405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:53719 (local address 127.9.39.190:34303)
0927 10:48:43.505544 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:43.505549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:43.506491 (+ 942us) server_negotiation.cc:407] Connection header received
0927 10:48:43.508100 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:43.508103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:43.508153 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:43.508241 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:43.509342 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.509967 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.511772 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.512066 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.512146 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.514946 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:43.514950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:43.514951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:43.514972 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:43.516401 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.516935 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.516943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.516944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.517015 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.517353 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.517356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.517358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.517529 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:43.517636 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:43.518185 (+ 549us) server_negotiation.cc:299] Negotiation successful
0927 10:48:43.518313 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:48:43.519408 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.505784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:34303 (local address 127.9.39.129:53719)
0927 10:48:43.506325 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:43.506379 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:43.507866 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:43.508399 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:43.508405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:43.508741 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:43.509187 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:43.509199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.510089 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.510096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:43.511649 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:43.511663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.512225 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.512230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:43.512444 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.513032 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:43.513058 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:43.514787 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:43.516517 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:43.516522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:43.516537 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:43.516828 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:43.517112 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:43.517115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:43.517117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:43.517253 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:43.517667 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:43.517674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:43.517947 (+ 273us) client_negotiation.cc:769] Sending connection context
0927 10:48:43.518159 (+ 212us) client_negotiation.cc:240] Negotiation successful
0927 10:48:43.518486 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":174,"threads_started":1}
I20260927 10:48:43.520691 12617 heartbeater.cc:347] Connected to a master server at 127.9.39.190:34303
I20260927 10:48:43.520936 12617 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:43.521413 12617 heartbeater.cc:511] Master 127.9.39.190:34303 requested a full tablet report, sending...
I20260927 10:48:43.523048 12426 ts_manager.cc:194] Registered new tserver with Master: c48410cc68a546a08c1b7c7650277c79 (127.9.39.129:35975)
I20260927 10:48:43.524161 12426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:53719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:43.533721 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34303
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:43.641198 12622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:43.641486 12622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:43.641568 12622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:43.645565 12622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:43.645649 12622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:43.645737 12622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:43.650964 12622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:43.651939 12622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:43.653076 12622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:43.662163 12630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:43.662165 12628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:43.662168 12627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.662583 12622 server_base.cc:1034] running on GCE node
I20260927 10:48:43.663048 12622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:43.663638 12622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:43.664817 12622 hybrid_clock.cc:648] HybridClock initialized: now 1790506123664807 us; error 32 us; skew 500 ppm
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:43.667372 12622 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:43.668278 12622 webserver.cc:460] Webserver started at http://127.9.39.130:34151/ using document root <none> and password file <none>
I20260927 10:48:43.668838 12622 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:43.668927 12622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:43.669144 12622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:43.671118 12622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "1f014b1636794c7a9b18c9c806972ca9"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "cb7016f6f9e85b55fd0a5a53200dfb41"
server_key_iv: "9986709874ef874bc5fb7179e4f29e2d"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.671656 12622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "1f014b1636794c7a9b18c9c806972ca9"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "cb7016f6f9e85b55fd0a5a53200dfb41"
server_key_iv: "9986709874ef874bc5fb7179e4f29e2d"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.675158 12622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:43.677366 12637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.678483 12622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:43.678606 12622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "1f014b1636794c7a9b18c9c806972ca9"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "cb7016f6f9e85b55fd0a5a53200dfb41"
server_key_iv: "9986709874ef874bc5fb7179e4f29e2d"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.678786 12622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:43.689006 12622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:43.691681 12622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:43.691877 12622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:43.692441 12622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:43.693212 12622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:43.693279 12622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.693342 12622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:43.693388 12622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.702209 12622 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:44051
I20260927 10:48:43.702229 12750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:44051 every 8 connection(s)
I20260927 10:48:43.703114 12622 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:43.709383 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12622
I20260927 10:48:43.709532 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:43.709822 9374 external_mini_cluster.cc:1428] Setting key e15a3cdcd3c2717fd72070790a27d16b
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:43.717372 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.704831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:60401 (local address 127.9.39.190:34303)
0927 10:48:43.704962 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:43.704965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:43.705743 (+ 778us) server_negotiation.cc:407] Connection header received
0927 10:48:43.707181 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:43.707185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:43.707244 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:43.707354 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:43.708510 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.709046 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.710336 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.710602 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.710723 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.713503 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:43.713506 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:43.713508 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:43.713530 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:43.715195 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.715672 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.715677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.715681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.715802 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.716136 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.716142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.716144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.716394 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:43.716528 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:43.717011 (+ 483us) server_negotiation.cc:299] Negotiation successful
0927 10:48:43.717150 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:48:43.718051 12753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.705150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:34303 (local address 127.9.39.130:60401)
0927 10:48:43.705603 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:43.705644 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:43.706967 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:43.707527 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:43.707534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:43.707886 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:43.708357 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:43.708371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.709195 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.709198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:43.710205 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:43.710215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.710861 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.710865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:43.711068 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.711684 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:43.711703 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:43.713306 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:43.715301 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:43.715306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:43.715317 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:43.715566 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:43.715903 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:43.715906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:43.715908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:43.716039 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:43.716543 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:43.716548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:43.716765 (+ 217us) client_negotiation.cc:769] Sending connection context
0927 10:48:43.716922 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 10:48:43.717156 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":111,"threads_started":1}
I20260927 10:48:43.719209 12751 heartbeater.cc:347] Connected to a master server at 127.9.39.190:34303
I20260927 10:48:43.719470 12751 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:48:43.719969 12751 heartbeater.cc:511] Master 127.9.39.190:34303 requested a full tablet report, sending...
I20260927 10:48:43.721019 12426 ts_manager.cc:194] Registered new tserver with Master: 1f014b1636794c7a9b18c9c806972ca9 (127.9.39.130:44051)
I20260927 10:48:43.721580 12426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:60401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:43.737799 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34303
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:43.847139 12756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:43.847383 12756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:43.847460 12756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:43.851539 12756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:43.851586 12756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:43.851682 12756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:43.856976 12756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:43.857928 12756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:43.859073 12756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:43.868198 12762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:43.868183 12764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.868321 12756 server_base.cc:1034] running on GCE node
W20260927 10:48:43.868180 12761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:43.868870 12756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:43.869429 12756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:43.870609 12756 hybrid_clock.cc:648] HybridClock initialized: now 1790506123870580 us; error 41 us; skew 500 ppm
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:43.873307 12756 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:43.874151 12756 webserver.cc:460] Webserver started at http://127.9.39.131:35015/ using document root <none> and password file <none>
I20260927 10:48:43.874665 12756 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:43.874791 12756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:43.874987 12756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:43.876838 12756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "64a886c9c29a4ee280783eb13ab876ec"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "dc909f55a2bf6ea339478ed5b554b0c2"
server_key_iv: "bfa635b2fd6bfbb8660fb9f29906ac13"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.877380 12756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "64a886c9c29a4ee280783eb13ab876ec"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "dc909f55a2bf6ea339478ed5b554b0c2"
server_key_iv: "bfa635b2fd6bfbb8660fb9f29906ac13"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.880998 12756 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 10:48:43.883370 12771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.884479 12756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:43.884599 12756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "64a886c9c29a4ee280783eb13ab876ec"
format_stamp: "Formatted at 2026-09-27 10:48:43 on dist-test-slave-p9q6"
server_key: "dc909f55a2bf6ea339478ed5b554b0c2"
server_key_iv: "bfa635b2fd6bfbb8660fb9f29906ac13"
server_key_version: "encryptionkey@0"
I20260927 10:48:43.884693 12756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:43.893759 12756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:43.896354 12756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:43.896555 12756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:43.897081 12756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:43.897845 12756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:43.897912 12756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.897979 12756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:43.898031 12756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:43.906626 12756 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:36161
I20260927 10:48:43.906730 12884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:36161 every 8 connection(s)
I20260927 10:48:43.907501 12756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:43.913763 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12756
I20260927 10:48:43.913848 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:43.914090 9374 external_mini_cluster.cc:1428] Setting key f6bab57f88954489136da4ff9f7e9ae8
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:43.920250 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.909150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:59325 (local address 127.9.39.190:34303)
0927 10:48:43.909317 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:43.909322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:43.910109 (+ 787us) server_negotiation.cc:407] Connection header received
0927 10:48:43.911426 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:43.911428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:43.911472 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:43.911544 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:43.912597 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.913110 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.914338 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.914606 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.914742 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.917342 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:43.917345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:43.917347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:43.917369 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:43.918545 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.919044 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.919046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.919047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.919118 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.919393 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.919395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.919396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.919547 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:43.919615 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:43.920041 (+ 426us) server_negotiation.cc:299] Negotiation successful
0927 10:48:43.920128 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 10:48:43.921005 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.909527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:34303 (local address 127.9.39.131:59325)
0927 10:48:43.909964 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:43.910003 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:43.911242 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:43.911696 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:43.911703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:43.912040 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:43.912476 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:43.912488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.913223 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.913226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:43.914219 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:43.914228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.914780 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.914784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:43.914999 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.915621 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:43.915639 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:43.917203 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:43.918639 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:43.918643 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:43.918654 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:43.918954 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:43.919190 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:43.919192 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:43.919194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:43.919312 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:43.919627 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:43.919631 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:43.919845 (+ 214us) client_negotiation.cc:769] Sending connection context
0927 10:48:43.919984 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 10:48:43.920201 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260927 10:48:43.922073 12885 heartbeater.cc:347] Connected to a master server at 127.9.39.190:34303
I20260927 10:48:43.922288 12885 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:43.922776 12885 heartbeater.cc:511] Master 127.9.39.190:34303 requested a full tablet report, sending...
I20260927 10:48:43.923688 12426 ts_manager.cc:194] Registered new tserver with Master: 64a886c9c29a4ee280783eb13ab876ec (127.9.39.131:36161)
I20260927 10:48:43.924202 12426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:59325
I20260927 10:48:43.927799 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 10:48:43 dist-test-slave-p9q6 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506123, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:43.952039 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:43.943859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.9.39.190:34303)
0927 10:48:43.943979 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:43.943983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:43.944117 (+ 134us) server_negotiation.cc:407] Connection header received
0927 10:48:43.944240 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:43.944242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:43.944289 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:43.944349 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:43.944992 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.945502 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.946244 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:43.946464 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:43.946537 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:43.948958 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:43.948961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:43.948963 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:43.948985 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:43.950472 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.950913 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.950915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.950917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.950990 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:43.951219 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:43.951221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:43.951222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:43.951465 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:43.951579 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:43.951754 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 10:48:43.951841 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 10:48:43.952409 12426 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46658
I20260927 10:48:43.953344 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12488
I20260927 10:48:43.958858 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12622
I20260927 10:48:43.964197 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12756
I20260927 10:48:43.969323 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12394
2026-09-27T10:48:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2071 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:44 dist-test-slave-p9q6 krb5kdc[12897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:44 dist-test-slave-p9q6 krb5kdc[12897](info): set up 2 sockets
Sep 27 10:48:44 dist-test-slave-p9q6 krb5kdc[12897](info): commencing operation
krb5kdc: starting...
W20260927 10:48:45.006742 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:45.125864 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37231
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:37231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:45.233704 12910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:45.234014 12910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:45.234090 12910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:45.238054 12910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:45.238139 12910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:45.238176 12910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:45.238214 12910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:45.238245 12910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:45.243444 12910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:37231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:45.244426 12910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:45.245409 12910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:45.253506 12916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:45.253547 12918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:45.253506 12915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:45.253908 12910 server_base.cc:1034] running on GCE node
I20260927 10:48:45.254356 12910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:45.255409 12910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:45.256589 12910 hybrid_clock.cc:648] HybridClock initialized: now 1790506125256570 us; error 42 us; skew 500 ppm
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:45.259184 12910 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:45.260008 12910 webserver.cc:460] Webserver started at http://127.9.39.190:41945/ using document root <none> and password file <none>
I20260927 10:48:45.260536 12910 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:45.260596 12910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:45.260815 12910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:45.262770 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "0ebdfb786fb24d1cb59afea66d240e5b"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "b8f440dc85fdfa1b286d6dd0e2cba467"
server_key_iv: "9bafda2f7295a430b9646e6eea384855"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.263331 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "0ebdfb786fb24d1cb59afea66d240e5b"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "b8f440dc85fdfa1b286d6dd0e2cba467"
server_key_iv: "9bafda2f7295a430b9646e6eea384855"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.266882 12910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:45.269168 12925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.270360 12910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:45.270478 12910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "0ebdfb786fb24d1cb59afea66d240e5b"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "b8f440dc85fdfa1b286d6dd0e2cba467"
server_key_iv: "9bafda2f7295a430b9646e6eea384855"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.270596 12910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:45.281383 12910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:45.284049 12910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:45.284262 12910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:45.291050 12910 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:37231
I20260927 10:48:45.291074 12977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:37231 every 8 connection(s)
I20260927 10:48:45.292012 12910 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:45.294467 12978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:45.299762 12978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b: Bootstrap starting.
I20260927 10:48:45.301494 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 12910
I20260927 10:48:45.301616 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:45.301900 9374 external_mini_cluster.cc:1428] Setting key 92de6af6afd7d031024747fac8e18e4d
I20260927 10:48:45.302349 12978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:45.303231 12978 log.cc:824] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:45.304947 12978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b: No bootstrap required, opened a new log
I20260927 10:48:45.308286 12978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37231 } }
I20260927 10:48:45.308571 12978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:45.308648 12978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ebdfb786fb24d1cb59afea66d240e5b, State: Initialized, Role: FOLLOWER
I20260927 10:48:45.309085 12978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37231 } }
I20260927 10:48:45.309208 12978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:45.309290 12978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:45.309401 12978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 0 FOLLOWER]: Advancing to term 1
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:45.310516 12978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37231 } }
I20260927 10:48:45.310905 12978 leader_election.cc:302] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ebdfb786fb24d1cb59afea66d240e5b; no voters:
I20260927 10:48:45.311224 12978 leader_election.cc:288] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:45.311334 12983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:45.311568 12983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [term 1 LEADER]: Becoming Leader. State: Replica: 0ebdfb786fb24d1cb59afea66d240e5b, State: Running, Role: LEADER
I20260927 10:48:45.311921 12983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37231 } }
I20260927 10:48:45.312512 12978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:45.313578 12985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ebdfb786fb24d1cb59afea66d240e5b. Latest consensus state: current_term: 1 leader_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37231 } } }
I20260927 10:48:45.313786 12985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:45.315056 12992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:45.315089 12984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebdfb786fb24d1cb59afea66d240e5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37231 } } }
I20260927 10:48:45.315189 12984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:45.316497 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:45.303045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.9.39.190:37231)
0927 10:48:45.303484 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:45.303500 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:45.303531 (+ 31us) server_negotiation.cc:407] Connection header received
0927 10:48:45.304127 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:45.304143 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:45.304486 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:45.304805 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:45.305576 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.306556 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.307414 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.307880 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.308070 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:45.311221 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:45.311230 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:45.311247 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:45.311279 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:45.313396 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:45.313940 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:45.313946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:45.313951 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:45.314093 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:45.314394 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:45.314397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:45.314399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:45.314936 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:45.315133 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:45.315527 (+ 394us) server_negotiation.cc:299] Negotiation successful
0927 10:48:45.315839 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":123,"threads_started":1}
I20260927 10:48:45.318019 12992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:45.323091 12992 catalog_manager.cc:1347] Generated new cluster ID: 66ca80b1e2d2491ea0ba16e0852658bf
I20260927 10:48:45.323160 12992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:45.345479 12992 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:45.346742 12992 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:45.356253 12992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ebdfb786fb24d1cb59afea66d240e5b: Generated new TSK 0
I20260927 10:48:45.356936 12992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:45.387738 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37231
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:45.495323 13004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:45.495595 13004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:45.495667 13004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:45.499779 13004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:45.499861 13004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:45.499956 13004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:45.505137 13004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:45.506168 13004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:45.507256 13004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:45.516548 13009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:45.516548 13012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:45.516572 13010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:45.516971 13004 server_base.cc:1034] running on GCE node
I20260927 10:48:45.517371 13004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:45.517997 13004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:45.519204 13004 hybrid_clock.cc:648] HybridClock initialized: now 1790506125519177 us; error 42 us; skew 500 ppm
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:45.521945 13004 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:45.522917 13004 webserver.cc:460] Webserver started at http://127.9.39.129:32993/ using document root <none> and password file <none>
I20260927 10:48:45.523464 13004 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:45.523551 13004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:45.523773 13004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:45.525709 13004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "55d5d03d5dbf47b2a5ea83c3f077e2e1"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "6e21759aec782b4e5fbecc06da279ab8"
server_key_iv: "0c2cf84be20fa5e67bbf7af7a6e84dfb"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.526245 13004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "55d5d03d5dbf47b2a5ea83c3f077e2e1"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "6e21759aec782b4e5fbecc06da279ab8"
server_key_iv: "0c2cf84be20fa5e67bbf7af7a6e84dfb"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.529811 13004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:48:45.532059 13019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.533215 13004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:48:45.533360 13004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "55d5d03d5dbf47b2a5ea83c3f077e2e1"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "6e21759aec782b4e5fbecc06da279ab8"
server_key_iv: "0c2cf84be20fa5e67bbf7af7a6e84dfb"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.533463 13004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:45.551396 13004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:45.554005 13004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:45.554188 13004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:45.554714 13004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:45.555461 13004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:45.555531 13004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.555598 13004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:45.555639 13004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.564782 13004 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:34143
I20260927 10:48:45.564810 13132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:34143 every 8 connection(s)
I20260927 10:48:45.565601 13004 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:45.573807 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13004
I20260927 10:48:45.573959 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:45.574278 9374 external_mini_cluster.cc:1428] Setting key 440b5fb0c65201647594e62cf00db092
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:45.579509 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:45.567418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:46281 (local address 127.9.39.190:37231)
0927 10:48:45.567575 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:45.567580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:45.568392 (+ 812us) server_negotiation.cc:407] Connection header received
0927 10:48:45.569739 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:45.569742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:45.569786 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:45.569858 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:45.571069 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.571606 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.572700 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.572942 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.573019 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:45.575883 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:45.575887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:45.575890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:45.575916 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:45.577302 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:45.577795 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:45.577800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:45.577801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:45.577883 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:45.578227 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:45.578230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:45.578232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:45.578500 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:45.578601 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:45.579121 (+ 520us) server_negotiation.cc:299] Negotiation successful
0927 10:48:45.579268 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 10:48:45.580169 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:45.567789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37231 (local address 127.9.39.129:46281)
0927 10:48:45.568244 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:45.568284 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:45.569518 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:45.570013 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:45.570019 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:45.570416 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:45.570943 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:45.570958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.571703 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.571706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:45.572611 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:45.572619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.573079 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.573082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:45.573288 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:45.573882 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:45.573897 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:45.575719 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:45.577413 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:45.577419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:45.577432 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:45.577683 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:45.577977 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:45.577981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:45.577982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:45.578114 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:45.578607 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:45.578612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:45.578890 (+ 278us) client_negotiation.cc:769] Sending connection context
0927 10:48:45.579047 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 10:48:45.579304 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260927 10:48:45.581403 13133 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37231
I20260927 10:48:45.581698 13133 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:45.582242 13133 heartbeater.cc:511] Master 127.9.39.190:37231 requested a full tablet report, sending...
I20260927 10:48:45.583798 12942 ts_manager.cc:194] Registered new tserver with Master: 55d5d03d5dbf47b2a5ea83c3f077e2e1 (127.9.39.129:34143)
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:48:45.584957 12942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:46281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:45.601171 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37231
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:45.708217 13138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:45.708451 13138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:45.708523 13138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:45.712637 13138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:45.712719 13138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:45.712809 13138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:45.718080 13138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:45.719123 13138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:45.720077 13138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:45.729117 13144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:45.729185 13143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:45.729241 13138 server_base.cc:1034] running on GCE node
W20260927 10:48:45.729178 13146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:45.729629 13138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:45.730260 13138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:45.731436 13138 hybrid_clock.cc:648] HybridClock initialized: now 1790506125731410 us; error 35 us; skew 500 ppm
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:45.734112 13138 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:45.735019 13138 webserver.cc:460] Webserver started at http://127.9.39.130:44975/ using document root <none> and password file <none>
I20260927 10:48:45.735556 13138 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:45.735621 13138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:45.735826 13138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:45.737917 13138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "dbf6e748b6cb40b2ad86509cffa4885c"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "d3fdf6c0ce918428e79a7982386d0845"
server_key_iv: "e6f392d86a32903628704ebc36befcf4"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.738499 13138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "dbf6e748b6cb40b2ad86509cffa4885c"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "d3fdf6c0ce918428e79a7982386d0845"
server_key_iv: "e6f392d86a32903628704ebc36befcf4"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.742130 13138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:45.744491 13153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.745638 13138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:45.745790 13138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "dbf6e748b6cb40b2ad86509cffa4885c"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "d3fdf6c0ce918428e79a7982386d0845"
server_key_iv: "e6f392d86a32903628704ebc36befcf4"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.745894 13138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:45.777009 13138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:45.780532 13138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:45.780735 13138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:45.781350 13138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:45.782183 13138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:45.782238 13138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.782323 13138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:45.782356 13138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.790894 13138 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:40559
I20260927 10:48:45.790964 13266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:40559 every 8 connection(s)
I20260927 10:48:45.791810 13138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:45.796927 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13138
I20260927 10:48:45.797070 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:45.797364 9374 external_mini_cluster.cc:1428] Setting key f9d7dceae4bbae02cdb053a81247226f
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:45.806629 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:45.793490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:47161 (local address 127.9.39.190:37231)
0927 10:48:45.793638 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:45.793644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:45.794458 (+ 814us) server_negotiation.cc:407] Connection header received
0927 10:48:45.795881 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:45.795884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:45.795928 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:45.796008 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:45.797308 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.797926 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.799448 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.799865 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.799990 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:45.802624 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:45.802630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:45.802636 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:45.802668 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:45.804424 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:45.804956 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:45.804962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:45.804966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:45.805061 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:45.805383 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:45.805388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:45.805392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:45.805710 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:45.805819 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:45.806273 (+ 454us) server_negotiation.cc:299] Negotiation successful
0927 10:48:45.806422 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:48:45.807351 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:45.793904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37231 (local address 127.9.39.130:47161)
0927 10:48:45.794314 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:45.794355 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:45.795667 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:45.796225 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:45.796232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:45.796565 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:45.797127 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:45.797165 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.798080 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.798083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:45.799322 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:45.799332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:45.800029 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:45.800032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:45.800178 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:45.800877 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:45.800896 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:45.802457 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:45.804557 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:45.804561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:45.804572 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:45.804852 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:45.805164 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:45.805167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:45.805169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:45.805292 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:45.805814 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:45.805819 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:45.806032 (+ 213us) client_negotiation.cc:769] Sending connection context
0927 10:48:45.806184 (+ 152us) client_negotiation.cc:240] Negotiation successful
0927 10:48:45.806439 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:48:45.808627 13267 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37231
I20260927 10:48:45.808936 13267 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:45.809425 13267 heartbeater.cc:511] Master 127.9.39.190:37231 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:48:45.810570 12942 ts_manager.cc:194] Registered new tserver with Master: dbf6e748b6cb40b2ad86509cffa4885c (127.9.39.130:40559)
I20260927 10:48:45.811156 12942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:47161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:45.826360 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37231
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:45.932237 13272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:45.932466 13272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:45.932507 13272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:45.936475 13272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:45.936527 13272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:45.936630 13272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:45.941764 13272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:45.942757 13272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:45.943758 13272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:45.953269 13278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:45.953280 13280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:45.953418 13272 server_base.cc:1034] running on GCE node
W20260927 10:48:45.953287 13277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:45.953994 13272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:45.954523 13272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:45.955719 13272 hybrid_clock.cc:648] HybridClock initialized: now 1790506125955721 us; error 53 us; skew 500 ppm
Sep 27 10:48:45 dist-test-slave-p9q6 krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:45.958381 13272 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:45.959263 13272 webserver.cc:460] Webserver started at http://127.9.39.131:46765/ using document root <none> and password file <none>
I20260927 10:48:45.959802 13272 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:45.959890 13272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:45.960105 13272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:45.962076 13272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "184a9953dcbb48989537bfb257040b2a"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "93a3e0bb93088e6cbf82a5376439a8a9"
server_key_iv: "c56ce6dce8f4ee6b095406d047dc28cb"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.962627 13272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "184a9953dcbb48989537bfb257040b2a"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "93a3e0bb93088e6cbf82a5376439a8a9"
server_key_iv: "c56ce6dce8f4ee6b095406d047dc28cb"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.966463 13272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 10:48:45.969246 13287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:45.970520 13272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:45.970685 13272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "184a9953dcbb48989537bfb257040b2a"
format_stamp: "Formatted at 2026-09-27 10:48:45 on dist-test-slave-p9q6"
server_key: "93a3e0bb93088e6cbf82a5376439a8a9"
server_key_iv: "c56ce6dce8f4ee6b095406d047dc28cb"
server_key_version: "encryptionkey@0"
I20260927 10:48:45.970870 13272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:45.997160 13272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:46.000417 13272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:46.000620 13272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:46.001217 13272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:46.002040 13272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:46.002110 13272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:46.002182 13272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:46.002230 13272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:46.011003 13272 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:45599
I20260927 10:48:46.011034 13400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:45599 every 8 connection(s)
I20260927 10:48:46.011878 13272 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:46.022169 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13272
I20260927 10:48:46.022312 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:46.022640 9374 external_mini_cluster.cc:1428] Setting key b989ca91b922a44695a88f1d4e138283
I20260927 10:48:46.026244 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:46.013785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:52989 (local address 127.9.39.190:37231)
0927 10:48:46.013930 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:46.013936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:46.014784 (+ 848us) server_negotiation.cc:407] Connection header received
0927 10:48:46.016487 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:46.016491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:46.016557 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:46.016666 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:46.018078 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.018636 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.019907 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.020134 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.020247 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:46.022665 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:46.022670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:46.022674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:46.022728 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:46.024396 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:46.024986 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:46.024989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:46.024991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:46.025068 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:46.025364 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:46.025366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:46.025368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:46.025526 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:46.025598 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:46.026029 (+ 431us) server_negotiation.cc:299] Negotiation successful
0927 10:48:46.026131 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:48:46.027120 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:46.014197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37231 (local address 127.9.39.131:52989)
0927 10:48:46.014616 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:46.014657 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:46.016246 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:46.016841 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:46.016848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:46.017202 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:46.017715 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:46.017728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.018804 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.018808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:46.019765 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:46.019780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.020285 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.020289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:46.020450 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:46.020980 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:46.020995 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:46.022500 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:46.024513 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:46.024519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:46.024532 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:46.024891 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:46.025160 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:46.025162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:46.025164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:46.025290 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:46.025610 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:46.025614 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:46.025833 (+ 219us) client_negotiation.cc:769] Sending connection context
0927 10:48:46.025981 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 10:48:46.026233 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260927 10:48:46.029273 13401 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37231
I20260927 10:48:46.029510 13401 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:46.029985 13401 heartbeater.cc:511] Master 127.9.39.190:37231 requested a full tablet report, sending...
I20260927 10:48:46.031004 12942 ts_manager.cc:194] Registered new tserver with Master: 184a9953dcbb48989537bfb257040b2a (127.9.39.131:45599)
I20260927 10:48:46.031527 12942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:52989
I20260927 10:48:46.036399 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:46.061851 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:46.052360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.9.39.129:34143)
0927 10:48:46.052667 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:46.052680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:46.052703 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:48:46.052751 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:46.052756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:46.052939 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:46.053079 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:46.053752 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.054538 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.055431 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.055815 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.055905 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:46.058463 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:46.058473 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:46.058487 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:46.058513 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:46.060117 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:46.060529 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:46.060534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:46.060536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:46.060613 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:46.060839 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:46.060842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:46.060843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:46.061110 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:46.061285 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:46.061448 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 10:48:46.061603 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":117,"threads_started":1}
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:46.072717 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:46.064657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52158 (local address 127.9.39.190:37231)
0927 10:48:46.064819 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:46.064824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:46.064861 (+ 37us) server_negotiation.cc:407] Connection header received
0927 10:48:46.065069 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:46.065072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:46.065120 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:46.065203 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:46.065799 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.066370 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.067137 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:46.067377 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:46.067472 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:46.069792 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:46.069795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:46.069797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:46.069821 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:46.071157 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:46.071541 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:46.071544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:46.071545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:46.071624 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:46.071853 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:46.071855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:46.071856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:46.072052 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:46.072151 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:46.072399 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 10:48:46.072554 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260927 10:48:46.073124 12942 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52158
I20260927 10:48:46.073725 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13004
I20260927 10:48:46.079761 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13138
I20260927 10:48:46.085718 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13272
I20260927 10:48:46.090974 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 12910
2026-09-27T10:48:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2121 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[13413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[13413](info): set up 2 sockets
Sep 27 10:48:46 dist-test-slave-p9q6 krb5kdc[13413](info): commencing operation
krb5kdc: starting...
W20260927 10:48:47.134795 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:47.251881 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46211
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:47.359748 13426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:47.360168 13426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:47.360255 13426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:47.364465 13426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:47.364542 13426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:47.364580 13426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:47.364621 13426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:47.364660 13426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:47.370015 13426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:47.371124 13426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:47.372207 13426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:47.380373 13431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:47.380493 13426 server_base.cc:1034] running on GCE node
W20260927 10:48:47.380430 13434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:47.380400 13432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:47.381314 13426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:47.382304 13426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:47.383486 13426 hybrid_clock.cc:648] HybridClock initialized: now 1790506127383456 us; error 42 us; skew 500 ppm
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:47.390125 13426 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:47.390942 13426 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:47.391007 13426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:47.391175 13426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:47.393189 13426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "adb5538626494c68beed917654520063"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "47d621002ed51d4991883dee6cf7c5d8"
server_key_iv: "660825ab4bac28e5827a78d8fcc1dc19"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.393714 13426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "adb5538626494c68beed917654520063"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "47d621002ed51d4991883dee6cf7c5d8"
server_key_iv: "660825ab4bac28e5827a78d8fcc1dc19"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.397351 13426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:48:47.399681 13440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.400848 13426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:47.400971 13426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "adb5538626494c68beed917654520063"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "47d621002ed51d4991883dee6cf7c5d8"
server_key_iv: "660825ab4bac28e5827a78d8fcc1dc19"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.401072 13426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:47.435084 13426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:47.438397 13426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:47.438617 13426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:47.445420 13426 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:46211
I20260927 10:48:47.445703 13492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:46211 every 8 connection(s)
I20260927 10:48:47.446079 13426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:47.447850 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13426
I20260927 10:48:47.447986 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:47.448285 9374 external_mini_cluster.cc:1428] Setting key 6dfc0b2a04ff3763bba217c446ddeff2
I20260927 10:48:47.448909 13493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:47.455010 13493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063: Bootstrap starting.
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:47.457149 13493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:47.458032 13493 log.cc:824] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:47.459884 13493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063: No bootstrap required, opened a new log
I20260927 10:48:47.461660 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:47.449478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.9.39.190:46211)
0927 10:48:47.449846 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:47.449860 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:47.449893 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:48:47.450543 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:47.450568 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:47.450969 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:47.451298 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:47.452080 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.453047 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.453875 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.454326 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.454491 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:47.456911 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:47.456915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:47.456928 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:47.456952 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:47.458870 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:47.459290 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:47.459296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:47.459302 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:47.459410 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:47.459699 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:47.459703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:47.459704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:47.460164 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:47.460364 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:47.460651 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 10:48:47.460925 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
I20260927 10:48:47.463516 13493 raft_consensus.cc:348] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb5538626494c68beed917654520063" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46211 } }
I20260927 10:48:47.463819 13493 raft_consensus.cc:374] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:47.463901 13493 raft_consensus.cc:729] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb5538626494c68beed917654520063, State: Initialized, Role: FOLLOWER
I20260927 10:48:47.464313 13493 consensus_queue.cc:260] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb5538626494c68beed917654520063" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46211 } }
I20260927 10:48:47.464455 13493 raft_consensus.cc:388] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:47.464538 13493 raft_consensus.cc:482] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:47.464643 13493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:47.465555 13493 raft_consensus.cc:504] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb5538626494c68beed917654520063" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46211 } }
I20260927 10:48:47.465893 13493 leader_election.cc:302] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adb5538626494c68beed917654520063; no voters:
I20260927 10:48:47.466264 13493 leader_election.cc:288] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:47.466349 13498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:47.466866 13498 raft_consensus.cc:686] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [term 1 LEADER]: Becoming Leader. State: Replica: adb5538626494c68beed917654520063, State: Running, Role: LEADER
I20260927 10:48:47.467172 13498 consensus_queue.cc:237] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb5538626494c68beed917654520063" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46211 } }
I20260927 10:48:47.467535 13493 sys_catalog.cc:564] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:47.468731 13499 sys_catalog.cc:455] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adb5538626494c68beed917654520063" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb5538626494c68beed917654520063" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46211 } } }
I20260927 10:48:47.468859 13499 sys_catalog.cc:458] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:47.469218 13507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:47.469223 13500 sys_catalog.cc:455] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adb5538626494c68beed917654520063. Latest consensus state: current_term: 1 leader_uuid: "adb5538626494c68beed917654520063" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb5538626494c68beed917654520063" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46211 } } }
I20260927 10:48:47.469372 13500 sys_catalog.cc:458] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:47.472033 13507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:47.477607 13507 catalog_manager.cc:1347] Generated new cluster ID: 379e1456e2174e5ba0fa5868d29b13cd
I20260927 10:48:47.477664 13507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:47.490677 13507 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:47.491882 13507 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:47.502367 13507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063: Generated new TSK 0
I20260927 10:48:47.503139 13507 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:47.533418 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46211
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:47.644407 13519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:47.644683 13519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:47.644728 13519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:47.648767 13519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:47.648821 13519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:47.648880 13519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:47.654037 13519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:47.655084 13519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:47.656023 13519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:47.665323 13525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:47.665323 13524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:47.665382 13519 server_base.cc:1034] running on GCE node
W20260927 10:48:47.665323 13527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:47.665992 13519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:47.666597 13519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:47.667804 13519 hybrid_clock.cc:648] HybridClock initialized: now 1790506127667792 us; error 28 us; skew 500 ppm
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:47.670480 13519 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:47.671193 13519 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:47.671253 13519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:47.671408 13519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:47.673286 13519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "99daf6d3726341e492eba79722350b03"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "24fc2e2fb6b45685f7739bddf95d852d"
server_key_iv: "88048180a1ff4b631cc490ac855965cb"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.673758 13519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "99daf6d3726341e492eba79722350b03"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "24fc2e2fb6b45685f7739bddf95d852d"
server_key_iv: "88048180a1ff4b631cc490ac855965cb"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.677184 13519 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:48:47.679417 13533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.680603 13519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:47.680703 13519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "99daf6d3726341e492eba79722350b03"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "24fc2e2fb6b45685f7739bddf95d852d"
server_key_iv: "88048180a1ff4b631cc490ac855965cb"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.680783 13519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:47.709226 13519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:47.712118 13519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:47.712260 13519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:47.712811 13519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:47.713564 13519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:47.713608 13519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.713646 13519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:47.713661 13519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.722277 13519 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:36797
I20260927 10:48:47.722294 13646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:36797 every 8 connection(s)
I20260927 10:48:47.722985 13519 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:47.729424 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13519
I20260927 10:48:47.729559 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:47.729869 9374 external_mini_cluster.cc:1428] Setting key 0ed604059c9e7cafdd59b1f7d377af07
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:47.738107 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:47.724754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:36725 (local address 127.9.39.190:46211)
0927 10:48:47.724888 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:47.724892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:47.725994 (+ 1102us) server_negotiation.cc:407] Connection header received
0927 10:48:47.726919 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:47.726922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:47.726978 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:47.727087 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:47.728403 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.728936 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.730536 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.730921 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.731027 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:47.733923 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:47.733926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:47.733929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:47.733954 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:47.735656 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:47.736278 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:47.736284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:47.736288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:47.736395 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:47.736757 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:47.736764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:47.736768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:47.737073 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:47.737202 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:47.737702 (+ 500us) server_negotiation.cc:299] Negotiation successful
0927 10:48:47.737863 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:48:47.738881 13649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:47.725074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46211 (local address 127.9.39.129:36725)
0927 10:48:47.725837 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:47.725879 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:47.726713 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:47.727261 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:47.727269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:47.727681 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:47.728262 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:47.728278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.729062 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.729065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:47.730409 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:47.730420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.731072 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.731078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:47.731296 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:47.732013 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:47.732039 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:47.733741 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:47.735776 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:47.735780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:47.735794 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:47.736127 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:47.736501 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:47.736503 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:47.736505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:47.736651 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:47.737184 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:47.737189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:47.737440 (+ 251us) client_negotiation.cc:769] Sending connection context
0927 10:48:47.737602 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 10:48:47.737883 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260927 10:48:47.740234 13647 heartbeater.cc:347] Connected to a master server at 127.9.39.190:46211
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:48:47.740484 13647 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:47.740960 13647 heartbeater.cc:511] Master 127.9.39.190:46211 requested a full tablet report, sending...
I20260927 10:48:47.742525 13457 ts_manager.cc:194] Registered new tserver with Master: 99daf6d3726341e492eba79722350b03 (127.9.39.129:36797)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:48:47.744129 13457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:36725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:47.757606 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46211
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:47.864843 13652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:47.865084 13652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:47.865135 13652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:47.869237 13652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:47.869288 13652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:47.869393 13652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:47.874521 13652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:47.875515 13652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:47.876466 13652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:47.885547 13660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:47.885540 13657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:47.885573 13652 server_base.cc:1034] running on GCE node
W20260927 10:48:47.885538 13658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:47.886157 13652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:47.886868 13652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:47.888044 13652 hybrid_clock.cc:648] HybridClock initialized: now 1790506127888028 us; error 28 us; skew 500 ppm
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:47.890826 13652 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:47.891510 13652 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:47.891601 13652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:47.891835 13652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:47.893846 13652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "622d308822684c1f9f53577e62f89dbd"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "c9983e6b568d7353f73693100a887bed"
server_key_iv: "29a4a8d5ecf98d37b70ba0fac39ed1a9"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.894397 13652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "622d308822684c1f9f53577e62f89dbd"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "c9983e6b568d7353f73693100a887bed"
server_key_iv: "29a4a8d5ecf98d37b70ba0fac39ed1a9"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.898393 13652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 10:48:47.900695 13666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.901832 13652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:47.901960 13652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "622d308822684c1f9f53577e62f89dbd"
format_stamp: "Formatted at 2026-09-27 10:48:47 on dist-test-slave-p9q6"
server_key: "c9983e6b568d7353f73693100a887bed"
server_key_iv: "29a4a8d5ecf98d37b70ba0fac39ed1a9"
server_key_version: "encryptionkey@0"
I20260927 10:48:47.902066 13652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:47.921078 13652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:47.924242 13652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:47.924438 13652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:47.924983 13652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:47.925761 13652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:47.925829 13652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.925897 13652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:47.925952 13652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:47.934365 13652 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:45455
I20260927 10:48:47.934381 13779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:45455 every 8 connection(s)
I20260927 10:48:47.934974 13652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
Sep 27 10:48:47 dist-test-slave-p9q6 krb5kdc[13413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:47.943646 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13652
I20260927 10:48:47.943768 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:47.944046 9374 external_mini_cluster.cc:1428] Setting key e3b214417ca75979dd1cb93a20a251c7
I20260927 10:48:47.947531 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:47.936554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:45725 (local address 127.9.39.190:46211)
0927 10:48:47.936689 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:47.936693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:47.937457 (+ 764us) server_negotiation.cc:407] Connection header received
0927 10:48:47.938625 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:47.938628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:47.938672 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:47.938777 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:47.939825 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.940329 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.941368 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.941580 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.941669 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:47.943998 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:47.944001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:47.944003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:47.944025 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:47.945536 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:47.946022 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:47.946027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:47.946029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:47.946106 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:47.946477 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:47.946480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:47.946482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:47.946659 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:47.946805 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:47.947264 (+ 459us) server_negotiation.cc:299] Negotiation successful
0927 10:48:47.947374 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 10:48:47.948521 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:47.936911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46211 (local address 127.9.39.130:45725)
0927 10:48:47.937320 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:47.937358 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:47.938411 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:47.938935 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:47.938941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:47.939273 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:47.939707 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:47.939719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.940426 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.940428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:47.941263 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:47.941271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:47.941700 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:47.941703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:47.941837 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:47.942312 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:47.942326 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:47.943826 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:47.945671 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:47.945676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:47.945688 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:47.945922 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:47.946221 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:47.946224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:47.946226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:47.946376 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:47.946803 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:47.946810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:47.947028 (+ 218us) client_negotiation.cc:769] Sending connection context
0927 10:48:47.947210 (+ 182us) client_negotiation.cc:240] Negotiation successful
0927 10:48:47.947521 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":104,"threads_started":1}
I20260927 10:48:47.949537 13780 heartbeater.cc:347] Connected to a master server at 127.9.39.190:46211
I20260927 10:48:47.949754 13780 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:47.950158 13780 heartbeater.cc:511] Master 127.9.39.190:46211 requested a full tablet report, sending...
I20260927 10:48:47.951200 13457 ts_manager.cc:194] Registered new tserver with Master: 622d308822684c1f9f53577e62f89dbd (127.9.39.130:45455)
I20260927 10:48:47.952062 13457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:45725
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:47.971881 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46211
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:48.079138 13785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:48.079375 13785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:48.079417 13785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:48.083316 13785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:48.083369 13785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:48.083429 13785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:48.088702 13785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:48.089656 13785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:48.090608 13785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:48.100291 13790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:48.100363 13791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.100473 13785 server_base.cc:1034] running on GCE node
W20260927 10:48:48.100308 13793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.101058 13785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:48.101615 13785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:48.102766 13785 hybrid_clock.cc:648] HybridClock initialized: now 1790506128102730 us; error 45 us; skew 500 ppm
Sep 27 10:48:48 dist-test-slave-p9q6 krb5kdc[13413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:48.105497 13785 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:48.106283 13785 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:48.106351 13785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:48.106511 13785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:48.108563 13785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "048d9996c93842b592fff84f50734aae"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "3b49a79b0bf5107722cf8d9c6490c893"
server_key_iv: "b6eef125ffa0f89245e7057001b6c79c"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.109086 13785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "048d9996c93842b592fff84f50734aae"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "3b49a79b0bf5107722cf8d9c6490c893"
server_key_iv: "b6eef125ffa0f89245e7057001b6c79c"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.112874 13785 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 10:48:48.115278 13799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.116606 13785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:48.116703 13785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "048d9996c93842b592fff84f50734aae"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "3b49a79b0bf5107722cf8d9c6490c893"
server_key_iv: "b6eef125ffa0f89245e7057001b6c79c"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.116796 13785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:48.151106 13785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:48.154074 13785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:48.154232 13785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:48.154790 13785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:48.155608 13785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:48.155654 13785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.155694 13785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:48.155709 13785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.164821 13785 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:37559
I20260927 10:48:48.164839 13912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:37559 every 8 connection(s)
I20260927 10:48:48.165436 13785 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:48.167848 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13785
I20260927 10:48:48.168005 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:48.168349 9374 external_mini_cluster.cc:1428] Setting key 11638db121df3a5d08e5a7b64ebae2b9
Sep 27 10:48:48 dist-test-slave-p9q6 krb5kdc[13413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:48.179920 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.167368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:52157 (local address 127.9.39.190:46211)
0927 10:48:48.167507 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.167511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.168620 (+ 1109us) server_negotiation.cc:407] Connection header received
0927 10:48:48.170282 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.170284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.170329 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.170398 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:48.171736 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.172252 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.173384 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.173590 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.173665 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.176128 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:48.176131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:48.176134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:48.176159 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:48.177773 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:48.178268 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:48.178271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:48.178272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:48.178345 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:48.178649 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:48.178651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:48.178652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:48.178953 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:48.179072 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.179610 (+ 538us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.179751 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:48:48.180843 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.167822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46211 (local address 127.9.39.131:52157)
0927 10:48:48.168413 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.168480 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.170053 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.170603 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.170615 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.171080 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:48.171596 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.171610 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.172365 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.172368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.173274 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.173282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.173717 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.173720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.173857 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.174409 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:48.174425 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:48.175972 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:48.177880 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:48.177885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:48.177899 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:48.178185 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:48.178419 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:48.178421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:48.178423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:48.178555 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:48.179070 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:48.179077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:48.179361 (+ 284us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.179556 (+ 195us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.179912 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":164,"threads_started":1}
I20260927 10:48:48.182133 13913 heartbeater.cc:347] Connected to a master server at 127.9.39.190:46211
I20260927 10:48:48.182413 13913 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:48.182932 13913 heartbeater.cc:511] Master 127.9.39.190:46211 requested a full tablet report, sending...
I20260927 10:48:48.183980 13456 ts_manager.cc:194] Registered new tserver with Master: 048d9996c93842b592fff84f50734aae (127.9.39.131:37559)
I20260927 10:48:48.184734 13456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:52157
I20260927 10:48:48.194514 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:48:48.204213 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.196560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.9.39.190:46211)
0927 10:48:48.196744 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.196750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.196884 (+ 134us) server_negotiation.cc:407] Connection header received
0927 10:48:48.197051 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.197056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.197105 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.197199 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:48.197921 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.198439 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.199375 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.199597 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.199727 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.201078 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:48.201082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:48.201085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:48.201117 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:48.202392 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:48.202897 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:48.202901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:48.202903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:48.202992 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:48.203284 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:48.203288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:48.203290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:48.203527 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:48.203684 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.203915 (+ 231us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.204037 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260927 10:48:48.207582 13456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:48.210556 13456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:48.224833 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.219235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.9.39.131:37559)
0927 10:48:48.219609 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.219622 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.219648 (+ 26us) server_negotiation.cc:407] Connection header received
0927 10:48:48.219753 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.219760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.219973 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.220147 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:48.220655 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.221767 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.223235 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.224081 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.224205 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.224367 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.224576 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.224667 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":155,"threads_started":1}
I20260927 10:48:48.224826 13924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.219046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:37559 (local address 127.0.0.1:60998)
0927 10:48:48.219399 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.219473 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.219600 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.220359 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.220366 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.220390 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:48.220525 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.220532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.221905 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.221909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.223091 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.223101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.224205 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.224209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.224372 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.224429 (+ 57us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.224546 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.224672 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":29,"thread_start_us":105,"threads_started":1}
I20260927 10:48:48.225391 13926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.219803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36797 (local address 127.0.0.1:56178)
0927 10:48:48.220041 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.220057 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.220163 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.221040 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.221043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.221060 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:48.221148 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.221154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.222601 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.222604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.223730 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.223739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.224997 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.225001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.225091 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.225112 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.225201 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.225266 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260927 10:48:48.225488 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.219978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56178 (local address 127.9.39.129:36797)
0927 10:48:48.220438 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.220446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.220467 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:48:48.220538 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.220543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.220757 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.220893 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:48.221316 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.222448 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.223855 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.224704 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.224799 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.224934 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.225239 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.225342 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":99,"threads_started":1}
I20260927 10:48:48.226138 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.220530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.9.39.130:45455)
0927 10:48:48.221353 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.221367 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.221386 (+ 19us) server_negotiation.cc:407] Connection header received
0927 10:48:48.221456 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.221460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.221650 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.221794 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:48.222035 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.223082 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.224782 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.225588 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.225702 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.225828 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.225918 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.225998 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":153,"threads_started":1}
I20260927 10:48:48.226086 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.220140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45455 (local address 127.0.0.1:32792)
0927 10:48:48.221014 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.221025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.221098 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.221792 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.221796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.221818 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:48.221923 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.221929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.223223 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.223226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.224630 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.224641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.225698 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.225702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.225793 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.225813 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.225893 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.225959 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":24,"spinlock_wait_cycles":1408,"thread_start_us":86,"threads_started":1}
I20260927 10:48:48.228181 13847 tablet_service.cc:1467] Processing CreateTablet for tablet 90fff305f7bc43dcb32169a72b4a8ba2 (DEFAULT_TABLE table=test-table [id=4644119a2c5641a68d31ac649e1f9fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:48.229225 13847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fff305f7bc43dcb32169a72b4a8ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:48.228983 13714 tablet_service.cc:1467] Processing CreateTablet for tablet 90fff305f7bc43dcb32169a72b4a8ba2 (DEFAULT_TABLE table=test-table [id=4644119a2c5641a68d31ac649e1f9fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:48.229837 13714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fff305f7bc43dcb32169a72b4a8ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:48.232543 13581 tablet_service.cc:1467] Processing CreateTablet for tablet 90fff305f7bc43dcb32169a72b4a8ba2 (DEFAULT_TABLE table=test-table [id=4644119a2c5641a68d31ac649e1f9fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:48.233606 13581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fff305f7bc43dcb32169a72b4a8ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:48.236202 13930 tablet_bootstrap.cc:492] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Bootstrap starting.
I20260927 10:48:48.237243 13931 tablet_bootstrap.cc:492] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Bootstrap starting.
I20260927 10:48:48.238646 13930 tablet_bootstrap.cc:654] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:48.239910 13931 tablet_bootstrap.cc:654] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:48.240028 13930 log.cc:824] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:48.240192 13932 tablet_bootstrap.cc:492] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: Bootstrap starting.
I20260927 10:48:48.240860 13931 log.cc:824] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:48.242268 13930 tablet_bootstrap.cc:492] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: No bootstrap required, opened a new log
I20260927 10:48:48.242306 13932 tablet_bootstrap.cc:654] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:48.242480 13930 ts_tablet_manager.cc:1397] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260927 10:48:48.242841 13931 tablet_bootstrap.cc:492] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: No bootstrap required, opened a new log
I20260927 10:48:48.243031 13931 ts_tablet_manager.cc:1397] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:48.243129 13932 log.cc:824] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:48.244977 13932 tablet_bootstrap.cc:492] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: No bootstrap required, opened a new log
I20260927 10:48:48.245168 13932 ts_tablet_manager.cc:1397] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:48:48.246047 13931 raft_consensus.cc:348] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.246317 13931 raft_consensus.cc:374] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:48.246398 13931 raft_consensus.cc:729] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622d308822684c1f9f53577e62f89dbd, State: Initialized, Role: FOLLOWER
I20260927 10:48:48.246313 13930 raft_consensus.cc:348] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.246625 13930 raft_consensus.cc:374] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:48.246712 13930 raft_consensus.cc:729] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048d9996c93842b592fff84f50734aae, State: Initialized, Role: FOLLOWER
I20260927 10:48:48.246917 13931 consensus_queue.cc:260] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.247145 13930 consensus_queue.cc:260] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.247566 13780 heartbeater.cc:503] Master 127.9.39.190:46211 was elected leader, sending a full tablet report...
I20260927 10:48:48.247929 13931 ts_tablet_manager.cc:1428] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:48:48.248162 13930 ts_tablet_manager.cc:1428] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 10:48:48.248530 13913 heartbeater.cc:503] Master 127.9.39.190:46211 was elected leader, sending a full tablet report...
I20260927 10:48:48.249365 13932 raft_consensus.cc:348] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.249619 13932 raft_consensus.cc:374] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:48.249707 13932 raft_consensus.cc:729] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99daf6d3726341e492eba79722350b03, State: Initialized, Role: FOLLOWER
I20260927 10:48:48.250082 13932 consensus_queue.cc:260] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.250763 13647 heartbeater.cc:503] Master 127.9.39.190:46211 was elected leader, sending a full tablet report...
I20260927 10:48:48.251107 13932 ts_tablet_manager.cc:1428] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260927 10:48:48.269459 13938 raft_consensus.cc:482] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:48.269709 13938 raft_consensus.cc:504] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.270913 13938 leader_election.cc:288] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 048d9996c93842b592fff84f50734aae (127.9.39.131:37559), 622d308822684c1f9f53577e62f89dbd (127.9.39.130:45455)
I20260927 10:48:48.275731 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.271779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:47955 (local address 127.9.39.130:45455)
0927 10:48:48.271923 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.271929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.272123 (+ 194us) server_negotiation.cc:407] Connection header received
0927 10:48:48.272258 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.272263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.272316 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.272388 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:48.272602 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.273321 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.274741 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.275165 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.275262 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.275301 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.275530 (+ 229us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.275580 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:48:48.275749 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.271632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45455 (local address 127.9.39.129:47955)
0927 10:48:48.272021 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.272034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.272123 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.272410 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.272412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.272426 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:48.272508 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.272511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.273415 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.273418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.274558 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.274565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.275298 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.275302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.275417 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.275440 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.275519 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.275591 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":84,"threads_started":1}
I20260927 10:48:48.275799 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.271218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:37559 (local address 127.9.39.129:56927)
0927 10:48:48.271541 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.271553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.271693 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.271997 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.272000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.272020 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:48.272163 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.272172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.273221 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.273224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.274538 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:48.274549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.275438 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.275441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:48.275547 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.275563 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.275627 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.275690 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":132,"threads_started":1}
I20260927 10:48:48.275880 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.271311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:56927 (local address 127.9.39.131:37559)
0927 10:48:48.271471 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.271476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.271646 (+ 170us) server_negotiation.cc:407] Connection header received
0927 10:48:48.271815 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.271820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.271875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.271964 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:48.272302 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.273113 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.274729 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.275185 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.275298 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.275348 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.275666 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.275764 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:48:48.276379 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fff305f7bc43dcb32169a72b4a8ba2" candidate_uuid: "99daf6d3726341e492eba79722350b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622d308822684c1f9f53577e62f89dbd" is_pre_election: true
I20260927 10:48:48.276487 13867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fff305f7bc43dcb32169a72b4a8ba2" candidate_uuid: "99daf6d3726341e492eba79722350b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d9996c93842b592fff84f50734aae" is_pre_election: true
I20260927 10:48:48.276726 13734 raft_consensus.cc:2454] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99daf6d3726341e492eba79722350b03 in term 0.
I20260927 10:48:48.276789 13867 raft_consensus.cc:2454] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99daf6d3726341e492eba79722350b03 in term 0.
I20260927 10:48:48.277221 13534 leader_election.cc:302] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 622d308822684c1f9f53577e62f89dbd, 99daf6d3726341e492eba79722350b03; no voters:
I20260927 10:48:48.277519 13938 raft_consensus.cc:2781] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:48.277596 13938 raft_consensus.cc:482] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:48.277635 13938 raft_consensus.cc:3037] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:48.278661 13938 raft_consensus.cc:504] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.279053 13938 leader_election.cc:288] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [CANDIDATE]: Term 1 election: Requested vote from peers 048d9996c93842b592fff84f50734aae (127.9.39.131:37559), 622d308822684c1f9f53577e62f89dbd (127.9.39.130:45455)
I20260927 10:48:48.279409 13867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fff305f7bc43dcb32169a72b4a8ba2" candidate_uuid: "99daf6d3726341e492eba79722350b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d9996c93842b592fff84f50734aae"
I20260927 10:48:48.279498 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fff305f7bc43dcb32169a72b4a8ba2" candidate_uuid: "99daf6d3726341e492eba79722350b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622d308822684c1f9f53577e62f89dbd"
I20260927 10:48:48.279508 13867 raft_consensus.cc:3037] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:48.279628 13734 raft_consensus.cc:3037] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:48.280689 13867 raft_consensus.cc:2454] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99daf6d3726341e492eba79722350b03 in term 1.
I20260927 10:48:48.280987 13534 leader_election.cc:302] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d9996c93842b592fff84f50734aae, 99daf6d3726341e492eba79722350b03; no voters:
I20260927 10:48:48.281071 13734 raft_consensus.cc:2454] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99daf6d3726341e492eba79722350b03 in term 1.
I20260927 10:48:48.281181 13938 raft_consensus.cc:2781] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:48.281420 13938 raft_consensus.cc:686] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 1 LEADER]: Becoming Leader. State: Replica: 99daf6d3726341e492eba79722350b03, State: Running, Role: LEADER
I20260927 10:48:48.281772 13938 consensus_queue.cc:237] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } }
I20260927 10:48:48.285030 13457 catalog_manager.cc:5495] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99daf6d3726341e492eba79722350b03 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "99daf6d3726341e492eba79722350b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99daf6d3726341e492eba79722350b03" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:48.304462 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.301080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.9.39.129:36797)
0927 10:48:48.301205 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.301209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.301222 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:48:48.301359 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.301362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.301424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.301522 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:48.301750 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.302243 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.303061 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:48.303271 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:48.303352 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:48.303615 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:48.304142 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:48.304240 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.304291 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.304345 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:48:48.310946 13734 raft_consensus.cc:1261] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 1 FOLLOWER]: Refusing update from remote peer 99daf6d3726341e492eba79722350b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:48.310947 13867 raft_consensus.cc:1261] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 1 FOLLOWER]: Refusing update from remote peer 99daf6d3726341e492eba79722350b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:48.311762 13941 consensus_queue.cc:1035] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "622d308822684c1f9f53577e62f89dbd" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:48.311998 13938 consensus_queue.cc:1035] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "048d9996c93842b592fff84f50734aae" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 37559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:48.319890 13946 mvcc.cc:205] Tried to move back new op lower bound from 7333913101553192960 to 7333913101445066752. Current Snapshot: MvccSnapshot[applied={T|T < 7333913101553192960}]
I20260927 10:48:48.320878 13945 mvcc.cc:205] Tried to move back new op lower bound from 7333913101553192960 to 7333913101445066752. Current Snapshot: MvccSnapshot[applied={T|T < 7333913101553192960}]
I20260927 10:48:48.328043 13455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:48.328238 13455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:48.330812 13455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adb5538626494c68beed917654520063: Sending DeleteTablet for 3 replicas of tablet 90fff305f7bc43dcb32169a72b4a8ba2
I20260927 10:48:48.331817 13714 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fff305f7bc43dcb32169a72b4a8ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:48 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:32792
I20260927 10:48:48.331838 13847 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fff305f7bc43dcb32169a72b4a8ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:48 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:60998
I20260927 10:48:48.332024 13581 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fff305f7bc43dcb32169a72b4a8ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:48 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:56178
I20260927 10:48:48.332422 13953 tablet_replica.cc:330] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: stopping tablet replica
I20260927 10:48:48.332434 13955 tablet_replica.cc:330] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03: stopping tablet replica
I20260927 10:48:48.332690 13954 tablet_replica.cc:330] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: stopping tablet replica
I20260927 10:48:48.332800 13955 raft_consensus.cc:2229] T 90fff305f7bc43dcb32169a72b4a8ba2 P 99daf6d3726341e492eba79722350b03 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:48.332847 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13519
I20260927 10:48:48.333165 13953 raft_consensus.cc:2229] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:48.333644 13954 raft_consensus.cc:2229] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:48.334033 13954 raft_consensus.cc:2258] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:48.334959 13953 raft_consensus.cc:2258] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:48.335698 13954 ts_tablet_manager.cc:1905] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:48.336417 13953 ts_tablet_manager.cc:1905] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:48.337950 13954 ts_tablet_manager.cc:1918] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:48.338063 13954 log.cc:1196] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/90fff305f7bc43dcb32169a72b4a8ba2
I20260927 10:48:48.338354 13954 ts_tablet_manager.cc:1939] T 90fff305f7bc43dcb32169a72b4a8ba2 P 622d308822684c1f9f53577e62f89dbd: Deleting consensus metadata
I20260927 10:48:48.339540 13441 catalog_manager.cc:4841] TS 622d308822684c1f9f53577e62f89dbd (127.9.39.130:45455): tablet 90fff305f7bc43dcb32169a72b4a8ba2 (table test-table [id=4644119a2c5641a68d31ac649e1f9fc6]) successfully deleted
I20260927 10:48:48.339738 13953 ts_tablet_manager.cc:1918] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:48.339839 13953 log.cc:1196] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/90fff305f7bc43dcb32169a72b4a8ba2
I20260927 10:48:48.340174 13953 ts_tablet_manager.cc:1939] T 90fff305f7bc43dcb32169a72b4a8ba2 P 048d9996c93842b592fff84f50734aae: Deleting consensus metadata
W20260927 10:48:48.341166 13441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:36797 (error 108)
I20260927 10:48:48.341635 13441 catalog_manager.cc:4841] TS 048d9996c93842b592fff84f50734aae (127.9.39.131:37559): tablet 90fff305f7bc43dcb32169a72b4a8ba2 (table test-table [id=4644119a2c5641a68d31ac649e1f9fc6]) successfully deleted
W20260927 10:48:48.342197 13928 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:48.341948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36797 (local address 127.0.0.1:56196)
0927 10:48:48.342063 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.342120 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36797: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 10:48:48.342401 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13652
W20260927 10:48:48.342367 13441 catalog_manager.cc:4584] TS 99daf6d3726341e492eba79722350b03 (127.9.39.129:36797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90fff305f7bc43dcb32169a72b4a8ba2: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36797: connect: Connection reset by peer (error 104)
I20260927 10:48:48.348383 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13785
I20260927 10:48:48.355922 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13426
2026-09-27T10:48:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2265 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T10:48:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:48Z Disabled control of system clock
I20260927 10:48:48.386421 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36823
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 10:48:48.495883 13960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:48.496137 13960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:48.496173 13960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:48.500123 13960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:48.500170 13960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:48.500185 13960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:48.500229 13960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:48.500259 13960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:48.504961 13960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:36823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:48.505998 13960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:48.507404 13960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:48.515698 13966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:48.515766 13965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:48.515698 13968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.516016 13960 server_base.cc:1034] running on GCE node
I20260927 10:48:48.516582 13960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:48.517624 13960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:48.518760 13960 hybrid_clock.cc:648] HybridClock initialized: now 1790506128518745 us; error 30 us; skew 500 ppm
I20260927 10:48:48.520592 13960 webserver.cc:460] Webserver started at http://127.9.39.190:35791/ using document root <none> and password file <none>
I20260927 10:48:48.521185 13960 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:48.521246 13960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:48.521476 13960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:48.523512 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "b4ac5487116e4daa902147e92ce2e117"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "a4addfe0fd092926ef07ba5a044e6dd9"
server_key_iv: "b5e668a2248cca6eb135fee0a4f798cf"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.524085 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "b4ac5487116e4daa902147e92ce2e117"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "a4addfe0fd092926ef07ba5a044e6dd9"
server_key_iv: "b5e668a2248cca6eb135fee0a4f798cf"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.527614 13960 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:48.529855 13974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.531075 13960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:48.531206 13960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "b4ac5487116e4daa902147e92ce2e117"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "a4addfe0fd092926ef07ba5a044e6dd9"
server_key_iv: "b5e668a2248cca6eb135fee0a4f798cf"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.531312 13960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:48.535696 13960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:48.536255 13960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:48.536432 13960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:48.543177 13960 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:36823
I20260927 10:48:48.543195 14026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:36823 every 8 connection(s)
I20260927 10:48:48.544116 13960 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:48.547215 14027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:48.552026 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 13960
I20260927 10:48:48.552155 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:48.552431 9374 external_mini_cluster.cc:1428] Setting key 8e87f5cad723030cc52d90702e6447f3
I20260927 10:48:48.552843 14027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117: Bootstrap starting.
I20260927 10:48:48.555356 14027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:48.556070 14027 log.cc:824] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:48.558131 14027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117: No bootstrap required, opened a new log
I20260927 10:48:48.558198 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.553884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.9.39.190:36823)
0927 10:48:48.554545 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.554559 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.554594 (+ 35us) server_negotiation.cc:407] Connection header received
0927 10:48:48.555333 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.555350 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.555754 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.556092 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:48.556904 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:48.556909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:48.556920 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:48.556931 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:48.556984 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:48.557001 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:48.557093 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.557231 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.557479 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":138,"threads_started":1}
I20260927 10:48:48.561795 14027 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ac5487116e4daa902147e92ce2e117" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36823 } }
I20260927 10:48:48.562004 14027 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:48.562089 14027 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ac5487116e4daa902147e92ce2e117, State: Initialized, Role: FOLLOWER
I20260927 10:48:48.562630 14027 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ac5487116e4daa902147e92ce2e117" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36823 } }
I20260927 10:48:48.562811 14027 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:48.562861 14027 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:48.562929 14027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:48.563742 14027 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ac5487116e4daa902147e92ce2e117" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36823 } }
I20260927 10:48:48.564019 14027 leader_election.cc:302] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4ac5487116e4daa902147e92ce2e117; no voters:
I20260927 10:48:48.564254 14027 leader_election.cc:288] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:48.564383 14032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:48.564633 14032 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [term 1 LEADER]: Becoming Leader. State: Replica: b4ac5487116e4daa902147e92ce2e117, State: Running, Role: LEADER
I20260927 10:48:48.564920 14032 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ac5487116e4daa902147e92ce2e117" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36823 } }
I20260927 10:48:48.565438 14027 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:48.566502 14033 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4ac5487116e4daa902147e92ce2e117" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ac5487116e4daa902147e92ce2e117" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36823 } } }
I20260927 10:48:48.566628 14033 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:48.566926 14034 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4ac5487116e4daa902147e92ce2e117. Latest consensus state: current_term: 1 leader_uuid: "b4ac5487116e4daa902147e92ce2e117" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ac5487116e4daa902147e92ce2e117" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36823 } } }
I20260927 10:48:48.567013 14034 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:48.567092 14041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:48.569949 14041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:48.574923 14041 catalog_manager.cc:1347] Generated new cluster ID: b1dee1719d4d43699bdd35335b5e7a8d
I20260927 10:48:48.574986 14041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:48.591006 14041 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:48.591343 14041 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:48.596418 14041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117: Generated new TSK 0
I20260927 10:48:48.597187 14041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:48:48.609486 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36823
--builtin_ntp_servers=127.9.39.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 10:48:48.719405 14051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:48.719658 14051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:48.719724 14051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:48.723847 14051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:48.723910 14051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:48.724002 14051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:48.728915 14051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:48.730113 14051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:48.731622 14051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:48.741358 14057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.741456 14051 server_base.cc:1034] running on GCE node
W20260927 10:48:48.741397 14059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:48.741358 14056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.742053 14051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:48.742802 14051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:48.743966 14051 hybrid_clock.cc:648] HybridClock initialized: now 1790506128743950 us; error 25 us; skew 500 ppm
I20260927 10:48:48.745814 14051 webserver.cc:460] Webserver started at http://127.9.39.129:44711/ using document root <none> and password file <none>
I20260927 10:48:48.746382 14051 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:48.746443 14051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:48.746682 14051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:48.748786 14051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "56ac025627424cb08c51702142b8c56c"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "d591cdc9354bd462ed479121eea4e9ef"
server_key_iv: "f22221264d9e34f36a24437a505c02db"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.749518 14051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "56ac025627424cb08c51702142b8c56c"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "d591cdc9354bd462ed479121eea4e9ef"
server_key_iv: "f22221264d9e34f36a24437a505c02db"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.753150 14051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260927 10:48:48.755885 14065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.757133 14051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260927 10:48:48.757227 14051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "56ac025627424cb08c51702142b8c56c"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "d591cdc9354bd462ed479121eea4e9ef"
server_key_iv: "f22221264d9e34f36a24437a505c02db"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.757377 14051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:48.761927 14051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:48.762502 14051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:48.762724 14051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:48.763278 14051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:48.764137 14051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:48.764181 14051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.764245 14051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:48.764276 14051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.773556 14051 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:36871
I20260927 10:48:48.773578 14178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:36871 every 8 connection(s)
I20260927 10:48:48.774523 14051 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:48.775590 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14051
I20260927 10:48:48.775717 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:48.776022 9374 external_mini_cluster.cc:1428] Setting key ffbbe7e31f61fe48c76dbb0bc48ec3c5
I20260927 10:48:48.778324 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36823
--builtin_ntp_servers=127.9.39.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 10:48:48.781786 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.776721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:60689 (local address 127.9.39.190:36823)
0927 10:48:48.776870 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.776875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.778282 (+ 1407us) server_negotiation.cc:407] Connection header received
0927 10:48:48.780052 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.780055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.780095 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.780166 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:48.781290 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:48.781294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:48.781296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:48.781308 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:48.781330 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:48.781339 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:48.781442 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.781596 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.781650 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:48:48.782631 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.777133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36823 (local address 127.9.39.129:60689)
0927 10:48:48.778139 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.778181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.779835 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.780344 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.780351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.780442 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:48.781020 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:48.781022 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:48.781078 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:48.781144 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:48.781435 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:48.781444 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:48.781450 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.781643 (+ 193us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.781881 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":130,"threads_started":1}
I20260927 10:48:48.783972 14179 heartbeater.cc:347] Connected to a master server at 127.9.39.190:36823
I20260927 10:48:48.784211 14179 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:48.784621 14179 heartbeater.cc:511] Master 127.9.39.190:36823 requested a full tablet report, sending...
I20260927 10:48:48.786126 13991 ts_manager.cc:194] Registered new tserver with Master: 56ac025627424cb08c51702142b8c56c (127.9.39.129:36871)
W20260927 10:48:48.887598 14182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:48.887876 14182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:48.887944 14182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:48.891904 14182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:48.891978 14182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:48.892063 14182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:48.896824 14182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:48.897823 14182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:48.899199 14182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:48.908205 14188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:48.908237 14187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.908308 14182 server_base.cc:1034] running on GCE node
W20260927 10:48:48.908200 14190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:48.908787 14182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:48.909462 14182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:48.910643 14182 hybrid_clock.cc:648] HybridClock initialized: now 1790506128910630 us; error 27 us; skew 500 ppm
I20260927 10:48:48.912410 14182 webserver.cc:460] Webserver started at http://127.9.39.130:43319/ using document root <none> and password file <none>
I20260927 10:48:48.912947 14182 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:48.913008 14182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:48.913173 14182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:48.915176 14182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "09a7cde56c5f4df8af088c349fb9462e"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "9cda3d9bd85da50777e39fb2bd6ac025"
server_key_iv: "5850ac2fb3285c0069dff209dedc8787"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.915654 14182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "09a7cde56c5f4df8af088c349fb9462e"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "9cda3d9bd85da50777e39fb2bd6ac025"
server_key_iv: "5850ac2fb3285c0069dff209dedc8787"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.919018 14182 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:48.921159 14196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.922329 14182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:48.922413 14182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "09a7cde56c5f4df8af088c349fb9462e"
format_stamp: "Formatted at 2026-09-27 10:48:48 on dist-test-slave-p9q6"
server_key: "9cda3d9bd85da50777e39fb2bd6ac025"
server_key_iv: "5850ac2fb3285c0069dff209dedc8787"
server_key_version: "encryptionkey@0"
I20260927 10:48:48.922493 14182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:48.926937 14182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:48.927453 14182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:48.927583 14182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:48.928020 14182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:48.928798 14182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:48.928841 14182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.928875 14182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:48.928890 14182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:48.937479 14182 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:46385
I20260927 10:48:48.937506 14309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:46385 every 8 connection(s)
I20260927 10:48:48.938304 14182 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:48.943668 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.939936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:36297 (local address 127.9.39.190:36823)
0927 10:48:48.940067 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:48.940070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:48.940955 (+ 885us) server_negotiation.cc:407] Connection header received
0927 10:48:48.942181 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:48.942184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:48.942220 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:48.942292 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:48.943327 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:48.943329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:48.943331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:48.943340 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:48.943355 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:48.943363 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:48.943429 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:48.943519 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 10:48:48.943551 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:48:48.944455 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:48.940253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36823 (local address 127.9.39.130:36297)
0927 10:48:48.940836 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:48.940872 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:48.941985 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:48.942425 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:48.942431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:48.942512 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:48.943103 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:48.943105 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:48.943142 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:48.943206 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:48.943416 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:48.943421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:48.943424 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 10:48:48.943535 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 10:48:48.943731 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":93,"threads_started":1}
I20260927 10:48:48.944830 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14182
I20260927 10:48:48.944971 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:48.945278 9374 external_mini_cluster.cc:1428] Setting key b6f017b1f2778f2d5dc9b5989740ea0f
I20260927 10:48:48.945602 14310 heartbeater.cc:347] Connected to a master server at 127.9.39.190:36823
I20260927 10:48:48.945822 14310 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:48.946282 14310 heartbeater.cc:511] Master 127.9.39.190:36823 requested a full tablet report, sending...
I20260927 10:48:48.947180 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36823
--builtin_ntp_servers=127.9.39.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 10:48:48.947322 13991 ts_manager.cc:194] Registered new tserver with Master: 09a7cde56c5f4df8af088c349fb9462e (127.9.39.130:46385)
W20260927 10:48:49.057525 14313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:49.057794 14313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:49.057878 14313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:49.061959 14313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:49.062014 14313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:49.062103 14313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:49.067265 14313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:49.068266 14313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:49.069535 14313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:49.078871 14321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:49.078848 14318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:49.079108 14313 server_base.cc:1034] running on GCE node
W20260927 10:48:49.079062 14319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:49.079658 14313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:49.080246 14313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:49.081446 14313 hybrid_clock.cc:648] HybridClock initialized: now 1790506129081423 us; error 31 us; skew 500 ppm
I20260927 10:48:49.083365 14313 webserver.cc:460] Webserver started at http://127.9.39.131:46845/ using document root <none> and password file <none>
I20260927 10:48:49.083928 14313 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:49.083998 14313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:49.084242 14313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:49.086326 14313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "9fde594a79e1e0015be3c89c1b8525f6"
server_key_iv: "8f88eb5d397314f4e8f18a2f1d62958d"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.086948 14313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "9fde594a79e1e0015be3c89c1b8525f6"
server_key_iv: "8f88eb5d397314f4e8f18a2f1d62958d"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.090559 14313 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260927 10:48:49.093132 14327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:49.094415 14313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260927 10:48:49.094592 14313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "9fde594a79e1e0015be3c89c1b8525f6"
server_key_iv: "8f88eb5d397314f4e8f18a2f1d62958d"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.094753 14313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:49.099426 14313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:49.100028 14313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:49.100210 14313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:49.100751 14313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:49.101639 14313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:49.101711 14313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:49.101781 14313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:49.101833 14313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:49.111629 14313 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:33477
I20260927 10:48:49.111662 14440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:33477 every 8 connection(s)
I20260927 10:48:49.112524 14313 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:49.113152 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14313
I20260927 10:48:49.113286 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:49.113575 9374 external_mini_cluster.cc:1428] Setting key b5f4736053cbca2b71c9e2b631af0fdc
I20260927 10:48:49.118683 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.114382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:60975 (local address 127.9.39.190:36823)
0927 10:48:49.114518 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.114522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.115838 (+ 1316us) server_negotiation.cc:407] Connection header received
0927 10:48:49.117178 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.117181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.117215 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.117290 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.118354 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.118357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.118359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.118368 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.118385 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.118392 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.118463 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.118553 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.118589 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:48:49.119514 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.115016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36823 (local address 127.9.39.131:60975)
0927 10:48:49.115695 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.115759 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.116972 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.117439 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.117445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.117527 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:49.118122 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:49.118123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:49.118163 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:49.118229 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:49.118451 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:49.118455 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:49.118458 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 10:48:49.118569 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 10:48:49.118814 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":132,"threads_started":1}
I20260927 10:48:49.120535 14441 heartbeater.cc:347] Connected to a master server at 127.9.39.190:36823
I20260927 10:48:49.120741 14441 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:49.121117 14441 heartbeater.cc:511] Master 127.9.39.190:36823 requested a full tablet report, sending...
I20260927 10:48:49.122008 13991 ts_manager.cc:194] Registered new tserver with Master: 0e87f3d2a9644bd298c2fdbf9f0d651c (127.9.39.131:33477)
I20260927 10:48:49.127710 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:48:49.131083 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.129605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.9.39.190:36823)
0927 10:48:49.129726 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.129729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.129898 (+ 169us) server_negotiation.cc:407] Connection header received
0927 10:48:49.129986 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.129988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.130018 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.130078 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.130802 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.130804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.130806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.130816 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.130834 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.130841 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.130914 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.130961 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.130990 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:48:49.133872 13991 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:48:49.136925 13991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:49.148118 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.144652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:46385 (local address 127.0.0.1:41290)
0927 10:48:49.144899 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.144950 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.145065 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.146119 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.146124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.146209 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:49.146801 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:49.146803 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:49.146822 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:49.146890 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:49.147764 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:49.147771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:49.147777 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 10:48:49.147839 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 10:48:49.147925 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"spinlock_wait_cycles":3584,"thread_start_us":77,"threads_started":1}
I20260927 10:48:49.148139 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.145220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.9.39.131:33477)
0927 10:48:49.145963 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.145973 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.145994 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:48:49.146043 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.146049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.146241 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.146996 (+ 755us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.147569 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.147573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.147587 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.147595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.147621 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.147635 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.147751 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.147890 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.147961 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":89,"threads_started":1}
I20260927 10:48:49.148355 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.144579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33477 (local address 127.0.0.1:57044)
0927 10:48:49.144886 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.144950 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.145065 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.146791 (+ 1726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.146794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.146816 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:49.147242 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:49.147247 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:49.147261 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:49.147293 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:49.147724 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:49.147733 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:49.147738 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:48:49.147843 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 10:48:49.147941 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":23,"spinlock_wait_cycles":6144,"thread_start_us":73,"threads_started":1}
I20260927 10:48:49.148458 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.145480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.9.39.129:36871)
0927 10:48:49.145750 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.145758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.146047 (+ 289us) server_negotiation.cc:407] Connection header received
0927 10:48:49.146260 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.146283 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.146449 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.146556 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.147834 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.147840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.147851 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.147865 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.147910 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.147949 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.148048 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.148216 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.148287 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260927 10:48:49.148844 14454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.145248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36871 (local address 127.0.0.1:42490)
0927 10:48:49.145954 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.145966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.146114 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.147197 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.147201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.147223 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:49.147643 (+ 420us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:49.147645 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:49.147655 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:49.147681 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:49.148088 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:49.148091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:49.148094 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 10:48:49.148698 (+ 604us) client_negotiation.cc:240] Negotiation successful
0927 10:48:49.148733 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":73,"threads_started":1}
I20260927 10:48:49.148978 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.145353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.9.39.130:46385)
0927 10:48:49.145671 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.145689 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.145710 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:48:49.145758 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.145763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.145944 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.146115 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.147271 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.147278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.147293 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.147307 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.147351 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.147651 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.148722 (+ 1071us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.148763 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.148822 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
I20260927 10:48:49.151443 14113 tablet_service.cc:1467] Processing CreateTablet for tablet b1feb3aeb6224e6bb80be3fdb3f0398e (DEFAULT_TABLE table=test-table [id=bdf71e701c294c2c9a2e370cd2e7d163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:49.151443 14375 tablet_service.cc:1467] Processing CreateTablet for tablet b1feb3aeb6224e6bb80be3fdb3f0398e (DEFAULT_TABLE table=test-table [id=bdf71e701c294c2c9a2e370cd2e7d163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:49.151468 14244 tablet_service.cc:1467] Processing CreateTablet for tablet b1feb3aeb6224e6bb80be3fdb3f0398e (DEFAULT_TABLE table=test-table [id=bdf71e701c294c2c9a2e370cd2e7d163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:49.152455 14244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1feb3aeb6224e6bb80be3fdb3f0398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:49.152472 14375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1feb3aeb6224e6bb80be3fdb3f0398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:49.152472 14113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1feb3aeb6224e6bb80be3fdb3f0398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:49.158079 14458 tablet_bootstrap.cc:492] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Bootstrap starting.
I20260927 10:48:49.159489 14459 tablet_bootstrap.cc:492] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: Bootstrap starting.
I20260927 10:48:49.160499 14458 tablet_bootstrap.cc:654] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:49.160725 14460 tablet_bootstrap.cc:492] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Bootstrap starting.
I20260927 10:48:49.161253 14458 log.cc:824] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:49.162530 14460 tablet_bootstrap.cc:654] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:49.162828 14459 tablet_bootstrap.cc:654] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:49.163259 14458 tablet_bootstrap.cc:492] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: No bootstrap required, opened a new log
I20260927 10:48:49.163386 14460 log.cc:824] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:49.163436 14458 ts_tablet_manager.cc:1397] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:48:49.163542 14459 log.cc:824] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:49.165232 14460 tablet_bootstrap.cc:492] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: No bootstrap required, opened a new log
I20260927 10:48:49.165232 14459 tablet_bootstrap.cc:492] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: No bootstrap required, opened a new log
I20260927 10:48:49.165436 14460 ts_tablet_manager.cc:1397] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:48:49.165436 14459 ts_tablet_manager.cc:1397] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260927 10:48:49.167526 14458 raft_consensus.cc:348] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.167758 14458 raft_consensus.cc:374] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:49.167817 14458 raft_consensus.cc:729] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a7cde56c5f4df8af088c349fb9462e, State: Initialized, Role: FOLLOWER
I20260927 10:48:49.168232 14458 consensus_queue.cc:260] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.168830 14310 heartbeater.cc:503] Master 127.9.39.190:36823 was elected leader, sending a full tablet report...
I20260927 10:48:49.169178 14458 ts_tablet_manager.cc:1428] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 10:48:49.169906 14460 raft_consensus.cc:348] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.169906 14459 raft_consensus.cc:348] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.170143 14460 raft_consensus.cc:374] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:49.170143 14459 raft_consensus.cc:374] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:49.170204 14459 raft_consensus.cc:729] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ac025627424cb08c51702142b8c56c, State: Initialized, Role: FOLLOWER
I20260927 10:48:49.170212 14460 raft_consensus.cc:729] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e87f3d2a9644bd298c2fdbf9f0d651c, State: Initialized, Role: FOLLOWER
I20260927 10:48:49.170639 14460 consensus_queue.cc:260] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.170639 14459 consensus_queue.cc:260] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.171469 14179 heartbeater.cc:503] Master 127.9.39.190:36823 was elected leader, sending a full tablet report...
I20260927 10:48:49.171558 14459 ts_tablet_manager.cc:1428] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260927 10:48:49.171558 14460 ts_tablet_manager.cc:1428] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:49.171989 14441 heartbeater.cc:503] Master 127.9.39.190:36823 was elected leader, sending a full tablet report...
W20260927 10:48:49.189144 14311 tablet.cc:2369] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:49.211041 14464 raft_consensus.cc:482] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:49.211274 14464 raft_consensus.cc:504] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.212406 14464 leader_election.cc:288] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e87f3d2a9644bd298c2fdbf9f0d651c (127.9.39.131:33477), 56ac025627424cb08c51702142b8c56c (127.9.39.129:36871)
I20260927 10:48:49.214272 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.212749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:51367 (local address 127.9.39.131:33477)
0927 10:48:49.212884 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.212891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.212906 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:48:49.213052 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.213056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.213093 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.213191 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.213883 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.213887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.213889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.213899 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.213916 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.213924 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.214012 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.214124 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.214158 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 10:48:49.214272 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.212685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33477 (local address 127.9.39.130:51367)
0927 10:48:49.212846 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.212861 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.212944 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.213207 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.213210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.213224 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:49.213723 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:49.213725 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:49.213736 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:49.213769 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:49.214007 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:49.214010 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:49.214013 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 10:48:49.214108 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 10:48:49.214143 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 10:48:49.214811 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.213162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:60913 (local address 127.9.39.129:36871)
0927 10:48:49.213282 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.213286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.213444 (+ 158us) server_negotiation.cc:407] Connection header received
0927 10:48:49.213569 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.213572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.213621 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.213715 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.214364 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.214369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.214371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.214383 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.214403 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.214413 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.214499 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.214619 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.214671 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 10:48:49.214826 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.212965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36871 (local address 127.9.39.130:60913)
0927 10:48:49.213359 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.213370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.213469 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.213732 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.213735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.213749 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:49.214199 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:48:49.214201 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:49.214211 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:48:49.214242 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:49.214541 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:49.214543 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:49.214546 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 10:48:49.214621 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 10:48:49.214652 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":125,"threads_started":1}
I20260927 10:48:49.214937 14395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1feb3aeb6224e6bb80be3fdb3f0398e" candidate_uuid: "09a7cde56c5f4df8af088c349fb9462e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" is_pre_election: true
I20260927 10:48:49.215257 14395 raft_consensus.cc:2454] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a7cde56c5f4df8af088c349fb9462e in term 0.
I20260927 10:48:49.215715 14197 leader_election.cc:302] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a7cde56c5f4df8af088c349fb9462e, 0e87f3d2a9644bd298c2fdbf9f0d651c; no voters:
I20260927 10:48:49.215958 14464 raft_consensus.cc:2781] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:49.215970 14133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1feb3aeb6224e6bb80be3fdb3f0398e" candidate_uuid: "09a7cde56c5f4df8af088c349fb9462e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56ac025627424cb08c51702142b8c56c" is_pre_election: true
I20260927 10:48:49.216044 14464 raft_consensus.cc:482] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:49.216104 14464 raft_consensus.cc:3037] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:49.216228 14133 raft_consensus.cc:2454] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a7cde56c5f4df8af088c349fb9462e in term 0.
I20260927 10:48:49.217247 14464 raft_consensus.cc:504] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.217598 14464 leader_election.cc:288] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [CANDIDATE]: Term 1 election: Requested vote from peers 0e87f3d2a9644bd298c2fdbf9f0d651c (127.9.39.131:33477), 56ac025627424cb08c51702142b8c56c (127.9.39.129:36871)
I20260927 10:48:49.217942 14395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1feb3aeb6224e6bb80be3fdb3f0398e" candidate_uuid: "09a7cde56c5f4df8af088c349fb9462e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c"
I20260927 10:48:49.218035 14395 raft_consensus.cc:3037] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:49.218034 14133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1feb3aeb6224e6bb80be3fdb3f0398e" candidate_uuid: "09a7cde56c5f4df8af088c349fb9462e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56ac025627424cb08c51702142b8c56c"
I20260927 10:48:49.218171 14133 raft_consensus.cc:3037] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:49.219266 14395 raft_consensus.cc:2454] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a7cde56c5f4df8af088c349fb9462e in term 1.
I20260927 10:48:49.219456 14133 raft_consensus.cc:2454] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a7cde56c5f4df8af088c349fb9462e in term 1.
I20260927 10:48:49.219583 14197 leader_election.cc:302] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a7cde56c5f4df8af088c349fb9462e, 0e87f3d2a9644bd298c2fdbf9f0d651c; no voters:
I20260927 10:48:49.219749 14464 raft_consensus.cc:2781] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:49.220008 14464 raft_consensus.cc:686] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 1 LEADER]: Becoming Leader. State: Replica: 09a7cde56c5f4df8af088c349fb9462e, State: Running, Role: LEADER
I20260927 10:48:49.220312 14464 consensus_queue.cc:237] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } }
I20260927 10:48:49.223419 13990 catalog_manager.cc:5495] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e reported cstate change: term changed from 0 to 1, leader changed from <none> to 09a7cde56c5f4df8af088c349fb9462e (127.9.39.130). New cstate: current_term: 1 leader_uuid: "09a7cde56c5f4df8af088c349fb9462e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09a7cde56c5f4df8af088c349fb9462e" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 46385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:49.244591 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.243138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.9.39.130:46385)
0927 10:48:49.243259 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.243264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.243278 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:48:49.243438 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.243440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.243495 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.243602 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.244211 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.244215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.244217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.244228 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.244244 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.244258 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.244346 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.244413 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.244450 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 10:48:49.251466 14133 raft_consensus.cc:1261] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 1 FOLLOWER]: Refusing update from remote peer 09a7cde56c5f4df8af088c349fb9462e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:49.251466 14395 raft_consensus.cc:1261] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 1 FOLLOWER]: Refusing update from remote peer 09a7cde56c5f4df8af088c349fb9462e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:49.252180 14468 consensus_queue.cc:1035] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [LEADER]: Connected to new peer: Peer: permanent_uuid: "56ac025627424cb08c51702142b8c56c" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:49.252357 14464 consensus_queue.cc:1035] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e87f3d2a9644bd298c2fdbf9f0d651c" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:49.260735 14472 mvcc.cc:205] Tried to move back new op lower bound from 7333913105405681664 to 7333913105288871936. Current Snapshot: MvccSnapshot[applied={T|T < 7333913105405681664}]
I20260927 10:48:49.261850 14474 mvcc.cc:205] Tried to move back new op lower bound from 7333913105405681664 to 7333913105288871936. Current Snapshot: MvccSnapshot[applied={T|T < 7333913105405681664}]
I20260927 10:48:49.269681 13989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:49.269835 13989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:49.272166 13989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4ac5487116e4daa902147e92ce2e117: Sending DeleteTablet for 3 replicas of tablet b1feb3aeb6224e6bb80be3fdb3f0398e
I20260927 10:48:49.272895 14375 tablet_service.cc:1514] Processing DeleteTablet for tablet b1feb3aeb6224e6bb80be3fdb3f0398e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:49 UTC) from {username='slave'} at 127.0.0.1:57044
I20260927 10:48:49.272948 14244 tablet_service.cc:1514] Processing DeleteTablet for tablet b1feb3aeb6224e6bb80be3fdb3f0398e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:49 UTC) from {username='slave'} at 127.0.0.1:41290
I20260927 10:48:49.273334 14480 tablet_replica.cc:330] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: stopping tablet replica
I20260927 10:48:49.273373 14113 tablet_service.cc:1514] Processing DeleteTablet for tablet b1feb3aeb6224e6bb80be3fdb3f0398e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:49 UTC) from {username='slave'} at 127.0.0.1:42490
I20260927 10:48:49.273597 14480 raft_consensus.cc:2229] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:49.273823 14482 tablet_replica.cc:330] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c: stopping tablet replica
I20260927 10:48:49.273895 14480 raft_consensus.cc:2258] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:49.274158 14482 raft_consensus.cc:2229] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:49.274441 14482 raft_consensus.cc:2258] T b1feb3aeb6224e6bb80be3fdb3f0398e P 56ac025627424cb08c51702142b8c56c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:49.274458 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14051
I20260927 10:48:49.278934 14481 tablet_replica.cc:330] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: stopping tablet replica
I20260927 10:48:49.279241 14481 raft_consensus.cc:2229] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:49.279635 14481 raft_consensus.cc:2258] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:49.280073 14480 ts_tablet_manager.cc:1905] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:49.280970 14481 ts_tablet_manager.cc:1905] T b1feb3aeb6224e6bb80be3fdb3f0398e P 09a7cde56c5f4df8af088c349fb9462e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 10:48:49.281929 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:36871 (error 108)
I20260927 10:48:49.282096 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14182
I20260927 10:48:49.283267 14480 ts_tablet_manager.cc:1918] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:49.283393 14480 log.cc:1196] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/b1feb3aeb6224e6bb80be3fdb3f0398e
I20260927 10:48:49.283654 14454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.283113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36871 (local address 127.0.0.1:42494)
0927 10:48:49.283462 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.283538 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241}
I20260927 10:48:49.283748 14480 ts_tablet_manager.cc:1939] T b1feb3aeb6224e6bb80be3fdb3f0398e P 0e87f3d2a9644bd298c2fdbf9f0d651c: Deleting consensus metadata
W20260927 10:48:49.284409 13975 catalog_manager.cc:4584] TS 56ac025627424cb08c51702142b8c56c (127.9.39.129:36871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1feb3aeb6224e6bb80be3fdb3f0398e: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36871: connect: Connection refused (error 111)
I20260927 10:48:49.285069 13975 catalog_manager.cc:4841] TS 0e87f3d2a9644bd298c2fdbf9f0d651c (127.9.39.131:33477): tablet b1feb3aeb6224e6bb80be3fdb3f0398e (table test-table [id=bdf71e701c294c2c9a2e370cd2e7d163]) successfully deleted
I20260927 10:48:49.288914 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14313
I20260927 10:48:49.288967 14454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.288759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:46385 (local address 127.0.0.1:41306)
0927 10:48:49.288876 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.288917 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.130:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 10:48:49.294459 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 13960
2026-09-27T10:48:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (936 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 10:48:49.315588 9374 mini_oidc.cc:152] Starting JWKS server
I20260927 10:48:49.317101 9374 webserver.cc:460] Webserver started at https://127.0.0.1:39935/ using document root <none> and password file <none>
I20260927 10:48:49.317277 9374 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 10:48:49.317534 9374 webserver.cc:460] Webserver started at http://127.0.0.1:44735/ using document root <none> and password file <none>
2026-09-27T10:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:49Z Disabled control of system clock
I20260927 10:48:49.326329 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36709
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:36709
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39935/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/testchainca.cert with env {}
W20260927 10:48:49.430158 14489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:49.430433 14489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:49.430501 14489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:49.434283 14489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:49.434327 14489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:49.434342 14489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:49.434358 14489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:49.434373 14489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 10:48:49.434388 14489 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 10:48:49.434402 14489 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39935/jwks.json
I20260927 10:48:49.439498 14489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:36709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39935/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:49.440378 14489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:49.441370 14489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:49.449174 14495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:49.449270 14489 server_base.cc:1034] running on GCE node
W20260927 10:48:49.449184 14497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:49.449208 14494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:49.449971 14489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:49.451009 14489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:49.452160 14489 hybrid_clock.cc:648] HybridClock initialized: now 1790506129452123 us; error 56 us; skew 500 ppm
I20260927 10:48:49.453809 14489 webserver.cc:460] Webserver started at http://127.9.39.190:37397/ using document root <none> and password file <none>
I20260927 10:48:49.454365 14489 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:49.454423 14489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:49.454628 14489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:49.456563 14489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "8f6c2664da7e4a7a97d1549b2501f369"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "1bec34c97d6e347c09b53f8f77bdd214"
server_key_iv: "405dce2904c5bdcf7f893a79b9029e38"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.457105 14489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "8f6c2664da7e4a7a97d1549b2501f369"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "1bec34c97d6e347c09b53f8f77bdd214"
server_key_iv: "405dce2904c5bdcf7f893a79b9029e38"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.460500 14489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260927 10:48:49.462831 14503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:49.464052 14489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:49.464211 14489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "8f6c2664da7e4a7a97d1549b2501f369"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "1bec34c97d6e347c09b53f8f77bdd214"
server_key_iv: "405dce2904c5bdcf7f893a79b9029e38"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.464318 14489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:49.476257 14489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:49.476890 14489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:49.477083 14489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:49.484033 14489 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:36709
I20260927 10:48:49.484055 14555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:36709 every 8 connection(s)
I20260927 10:48:49.484930 14489 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:49.488217 14556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:49.491530 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14489
I20260927 10:48:49.491672 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:49.491979 9374 external_mini_cluster.cc:1428] Setting key 31c61ee357441e56239f15a55d97f83e
I20260927 10:48:49.494354 14556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369: Bootstrap starting.
I20260927 10:48:49.496579 14556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:49.497356 14556 log.cc:824] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:49.499346 14556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369: No bootstrap required, opened a new log
I20260927 10:48:49.501009 14559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.493393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.9.39.190:36709)
0927 10:48:49.494007 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.494021 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.494053 (+ 32us) server_negotiation.cc:407] Connection header received
0927 10:48:49.494770 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.494787 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.495190 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.495507 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.495941 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.496950 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.497867 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.498330 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.498513 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:49.499383 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.499387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.499399 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.499808 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.499860 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.499879 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.499989 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.500135 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.500367 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":128,"threads_started":1}
I20260927 10:48:49.503288 14556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6c2664da7e4a7a97d1549b2501f369" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36709 } }
I20260927 10:48:49.503580 14556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:49.503635 14556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6c2664da7e4a7a97d1549b2501f369, State: Initialized, Role: FOLLOWER
I20260927 10:48:49.503944 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:48:49.504093 14556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6c2664da7e4a7a97d1549b2501f369" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36709 } }
I20260927 10:48:49.504264 14556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:49.504334 14556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:49.504436 14556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:49.505564 14556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6c2664da7e4a7a97d1549b2501f369" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36709 } }
I20260927 10:48:49.505903 14556 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6c2664da7e4a7a97d1549b2501f369; no voters:
I20260927 10:48:49.506165 14556 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:49.506314 14561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:49.506579 14561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [term 1 LEADER]: Becoming Leader. State: Replica: 8f6c2664da7e4a7a97d1549b2501f369, State: Running, Role: LEADER
I20260927 10:48:49.506904 14561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6c2664da7e4a7a97d1549b2501f369" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36709 } }
I20260927 10:48:49.507431 14556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:49.508407 14563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6c2664da7e4a7a97d1549b2501f369. Latest consensus state: current_term: 1 leader_uuid: "8f6c2664da7e4a7a97d1549b2501f369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6c2664da7e4a7a97d1549b2501f369" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36709 } } }
I20260927 10:48:49.508566 14563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:49.508899 14570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:49.509603 14562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6c2664da7e4a7a97d1549b2501f369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6c2664da7e4a7a97d1549b2501f369" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36709 } } }
I20260927 10:48:49.509717 14562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:49.511927 14570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:49.517544 14570 catalog_manager.cc:1347] Generated new cluster ID: e87439db1c254ab6873bc6271baafef4
I20260927 10:48:49.517650 14570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:49.539042 14570 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:49.539986 14570 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:49.549384 14570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6c2664da7e4a7a97d1549b2501f369: Generated new TSK 0
I20260927 10:48:49.550040 14570 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 10:48:49.584708 14586 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 10:48:49.584817 14586 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 10:48:49.585222 14559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.562491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.9.39.190:36709)
0927 10:48:49.562630 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.562637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.562878 (+ 241us) server_negotiation.cc:407] Connection header received
0927 10:48:49.563115 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.563124 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.563169 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.563266 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0927 10:48:49.563537 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.564056 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.565039 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.565269 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.565374 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:49.565717 (+ 343us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 10:48:49.584826 (+ 19109us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 10:48:49.585091 (+ 265us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39606: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260927 10:48:49.585314 14559 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39606: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 10:48:49.586386 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:49.562431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36709 (local address 127.0.0.1:39606)
0927 10:48:49.562799 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.562811 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.562945 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.563291 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.563294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.563302 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0927 10:48:49.563430 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:49.563435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.564193 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.564198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:49.564917 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:49.564925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.565389 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.565393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:49.565490 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:49.565528 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 10:48:49.585067 (+ 19539us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 10:48:49.585291 (+ 224us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.39.190:36709: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.39.190:36709: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 10:48:49.587096 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14489
2026-09-27T10:48:49Z chronyd exiting
I20260927 10:48:49.623924 9374 test_util.cc:175] -----------------------------------------------
I20260927 10:48:49.623998 9374 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790506104903010-9374-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 10:48:49.624935 9374 mini_oidc.cc:152] Starting JWKS server
I20260927 10:48:49.625720 9374 webserver.cc:460] Webserver started at https://127.0.0.1:43141/ using document root <none> and password file <none>
I20260927 10:48:49.625800 9374 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 10:48:49.626034 9374 webserver.cc:460] Webserver started at http://127.0.0.1:36939/ using document root <none> and password file <none>
2026-09-27T10:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:49Z Disabled control of system clock
I20260927 10:48:49.634656 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39865
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39865
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43141/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/testchainca.cert with env {}
W20260927 10:48:49.740634 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:49.740871 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:49.740909 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:49.744810 14592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:49.744858 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:49.744874 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:49.744889 14592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:49.744904 14592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 10:48:49.744925 14592 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 10:48:49.744942 14592 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43141/jwks.json
I20260927 10:48:49.750553 14592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43141/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:49.751499 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:49.752386 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:49.760286 14600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:49.760389 14597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:49.760352 14598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:49.760510 14592 server_base.cc:1034] running on GCE node
I20260927 10:48:49.761098 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:49.762133 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:49.763315 14592 hybrid_clock.cc:648] HybridClock initialized: now 1790506129763296 us; error 28 us; skew 500 ppm
I20260927 10:48:49.765281 14592 webserver.cc:460] Webserver started at http://127.9.39.190:45851/ using document root <none> and password file <none>
I20260927 10:48:49.765831 14592 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:49.765888 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:49.766115 14592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:49.768172 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "74cbffa0b9354b948e924a953f523dc7"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "188b305590eeb5e678293034402336e2"
server_key_iv: "5184122ee65a619b80449d14aee69485"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.768736 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "74cbffa0b9354b948e924a953f523dc7"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "188b305590eeb5e678293034402336e2"
server_key_iv: "5184122ee65a619b80449d14aee69485"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.772296 14592 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:48:49.774587 14606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:49.775792 14592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:49.775938 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "74cbffa0b9354b948e924a953f523dc7"
format_stamp: "Formatted at 2026-09-27 10:48:49 on dist-test-slave-p9q6"
server_key: "188b305590eeb5e678293034402336e2"
server_key_iv: "5184122ee65a619b80449d14aee69485"
server_key_version: "encryptionkey@0"
I20260927 10:48:49.776058 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:49.787295 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:49.787880 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:49.788058 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:49.794401 14592 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39865
I20260927 10:48:49.794421 14658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39865 every 8 connection(s)
I20260927 10:48:49.795321 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:49.798141 14659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:49.799525 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14592
I20260927 10:48:49.799642 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:49.799948 9374 external_mini_cluster.cc:1428] Setting key 32a11a7fbac49fcc52031a1e6a091cc8
I20260927 10:48:49.804070 14659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7: Bootstrap starting.
I20260927 10:48:49.806105 14659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:49.806877 14659 log.cc:824] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:49.808699 14659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7: No bootstrap required, opened a new log
I20260927 10:48:49.808779 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.801191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.9.39.190:39865)
0927 10:48:49.801818 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.801831 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.801864 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:48:49.802468 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.802483 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.802929 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.803245 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:49.803648 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.804634 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.805519 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.805979 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.806152 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:49.807027 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:49.807032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:49.807043 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:49.807521 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:49.807601 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:49.807622 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:49.807733 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:49.807896 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 10:48:49.808134 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":130,"threads_started":1}
I20260927 10:48:49.811468 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:48:49.812691 14659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cbffa0b9354b948e924a953f523dc7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39865 } }
I20260927 10:48:49.812933 14659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:49.813016 14659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cbffa0b9354b948e924a953f523dc7, State: Initialized, Role: FOLLOWER
I20260927 10:48:49.813443 14659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cbffa0b9354b948e924a953f523dc7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39865 } }
I20260927 10:48:49.813587 14659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:49.813650 14659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:49.813743 14659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:49.814512 14659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cbffa0b9354b948e924a953f523dc7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39865 } }
I20260927 10:48:49.814847 14659 leader_election.cc:302] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74cbffa0b9354b948e924a953f523dc7; no voters:
I20260927 10:48:49.815137 14659 leader_election.cc:288] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:49.815253 14664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:49.815481 14664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [term 1 LEADER]: Becoming Leader. State: Replica: 74cbffa0b9354b948e924a953f523dc7, State: Running, Role: LEADER
I20260927 10:48:49.815775 14664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cbffa0b9354b948e924a953f523dc7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39865 } }
I20260927 10:48:49.816319 14659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:49.817633 14666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74cbffa0b9354b948e924a953f523dc7. Latest consensus state: current_term: 1 leader_uuid: "74cbffa0b9354b948e924a953f523dc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cbffa0b9354b948e924a953f523dc7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39865 } } }
I20260927 10:48:49.817643 14665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74cbffa0b9354b948e924a953f523dc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cbffa0b9354b948e924a953f523dc7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39865 } } }
I20260927 10:48:49.817826 14665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:49.818073 14666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:49.818576 14674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:49.821031 14674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:49.825814 14674 catalog_manager.cc:1347] Generated new cluster ID: 0e0d2bb9e24c4ce5803f1f1dea3948be
I20260927 10:48:49.825874 14674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:49.871405 14674 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:49.872355 14674 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:49.877365 14674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74cbffa0b9354b948e924a953f523dc7: Generated new TSK 0
I20260927 10:48:49.878021 14674 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 10:48:49.909555 14689 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 10:48:49.909660 14689 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 10:48:49.910259 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:49.895127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.9.39.190:39865)
0927 10:48:49.895265 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:49.895270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:49.895419 (+ 149us) server_negotiation.cc:407] Connection header received
0927 10:48:49.895557 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:49.895561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:49.895616 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:49.895725 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0927 10:48:49.895923 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.896702 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.897524 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.897853 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.897952 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:49.898214 (+ 262us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 10:48:49.909878 (+ 11664us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 10:48:49.910130 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47118: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260927 10:48:49.910322 14662 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47118: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260927 10:48:49.910300 14688 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:49.895002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39865 (local address 127.0.0.1:47118)
0927 10:48:49.895331 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:49.895344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:49.895434 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:49.895726 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:49.895728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:49.895734 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0927 10:48:49.895821 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:49.895825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.896816 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.896818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:49.897411 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:49.897418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:49.897959 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:49.897961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:49.898021 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:49.898040 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 10:48:49.910042 (+ 12002us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 10:48:49.910175 (+ 133us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.39.190:39865: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260927 10:48:49.910979 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14592
2026-09-27T10:48:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (325 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 10:48:49.950068 9374 mini_oidc.cc:152] Starting JWKS server
I20260927 10:48:49.950990 9374 webserver.cc:460] Webserver started at https://127.0.0.1:37369/ using document root <none> and password file <none>
I20260927 10:48:49.951071 9374 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 10:48:49.951279 9374 webserver.cc:460] Webserver started at http://127.0.0.1:33015/ using document root <none> and password file <none>
2026-09-27T10:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:49Z Disabled control of system clock
I20260927 10:48:49.959714 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36241
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:36241
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37369/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 10:48:50.065973 14695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:50.066205 14695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:50.066258 14695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:50.070194 14695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:50.070245 14695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:50.070258 14695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:50.070274 14695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:50.070317 14695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 10:48:50.070348 14695 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 10:48:50.070369 14695 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37369/jwks.json
I20260927 10:48:50.075181 14695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:36241
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37369/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:50.076192 14695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:50.077492 14695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:48:50.085824 14695 server_base.cc:1034] running on GCE node
W20260927 10:48:50.085794 14700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:50.085780 14701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:50.085780 14703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:50.086745 14695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:50.087858 14695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:50.089061 14695 hybrid_clock.cc:648] HybridClock initialized: now 1790506130089036 us; error 52 us; skew 500 ppm
I20260927 10:48:50.091023 14695 webserver.cc:460] Webserver started at http://127.9.39.190:36757/ using document root <none> and password file <none>
I20260927 10:48:50.091600 14695 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:50.091701 14695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:50.091964 14695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:50.094105 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "2aadecf324b843b99ffd8158a35750fa"
format_stamp: "Formatted at 2026-09-27 10:48:50 on dist-test-slave-p9q6"
server_key: "ffe15957cfee9bf93f84dcfe961c401d"
server_key_iv: "29c87d3617341951fd879ffb692f0557"
server_key_version: "encryptionkey@0"
I20260927 10:48:50.094681 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "2aadecf324b843b99ffd8158a35750fa"
format_stamp: "Formatted at 2026-09-27 10:48:50 on dist-test-slave-p9q6"
server_key: "ffe15957cfee9bf93f84dcfe961c401d"
server_key_iv: "29c87d3617341951fd879ffb692f0557"
server_key_version: "encryptionkey@0"
I20260927 10:48:50.098345 14695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:50.100811 14709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:50.102026 14695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:50.102191 14695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "2aadecf324b843b99ffd8158a35750fa"
format_stamp: "Formatted at 2026-09-27 10:48:50 on dist-test-slave-p9q6"
server_key: "ffe15957cfee9bf93f84dcfe961c401d"
server_key_iv: "29c87d3617341951fd879ffb692f0557"
server_key_version: "encryptionkey@0"
I20260927 10:48:50.102310 14695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:50.113858 14695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:50.114445 14695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:50.114634 14695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:50.121591 14761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:36241 every 8 connection(s)
I20260927 10:48:50.121588 14695 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:36241
I20260927 10:48:50.122526 14695 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:50.124706 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14695
I20260927 10:48:50.124825 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:50.125092 9374 external_mini_cluster.cc:1428] Setting key d5cb737de5c4b1d315aef6d4bc366a37
I20260927 10:48:50.125808 14762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:50.131457 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa: Bootstrap starting.
I20260927 10:48:50.133670 14762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:50.133872 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:50.126427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.9.39.190:36241)
0927 10:48:50.127113 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:50.127128 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:50.127162 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:48:50.127816 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:50.127831 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:50.128205 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:50.128529 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:50.128931 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.129835 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.130742 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.131227 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.131398 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:50.132249 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:50.132254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:50.132265 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:50.132619 (+ 354us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:50.132672 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:50.132691 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:50.132807 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:50.132953 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 10:48:50.133185 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":141,"threads_started":1}
I20260927 10:48:50.134436 14762 log.cc:824] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:50.136363 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa: No bootstrap required, opened a new log
I20260927 10:48:50.136404 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:48:50.139827 14762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadecf324b843b99ffd8158a35750fa" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36241 } }
I20260927 10:48:50.140058 14762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:50.140136 14762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aadecf324b843b99ffd8158a35750fa, State: Initialized, Role: FOLLOWER
I20260927 10:48:50.140549 14762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadecf324b843b99ffd8158a35750fa" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36241 } }
I20260927 10:48:50.140692 14762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:50.140777 14762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:50.140893 14762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:50.141709 14762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadecf324b843b99ffd8158a35750fa" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36241 } }
I20260927 10:48:50.142028 14762 leader_election.cc:302] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aadecf324b843b99ffd8158a35750fa; no voters:
I20260927 10:48:50.142318 14762 leader_election.cc:288] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:50.142434 14767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:50.142650 14767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [term 1 LEADER]: Becoming Leader. State: Replica: 2aadecf324b843b99ffd8158a35750fa, State: Running, Role: LEADER
I20260927 10:48:50.142985 14767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadecf324b843b99ffd8158a35750fa" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36241 } }
I20260927 10:48:50.143486 14762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:50.144541 14768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aadecf324b843b99ffd8158a35750fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadecf324b843b99ffd8158a35750fa" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36241 } } }
I20260927 10:48:50.144549 14769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aadecf324b843b99ffd8158a35750fa. Latest consensus state: current_term: 1 leader_uuid: "2aadecf324b843b99ffd8158a35750fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadecf324b843b99ffd8158a35750fa" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36241 } } }
I20260927 10:48:50.144688 14769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:50.144688 14768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:50.145175 14776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:50.147686 14776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:50.152529 14776 catalog_manager.cc:1347] Generated new cluster ID: e139fd22ed814c668ee2827fc9354fbf
I20260927 10:48:50.152586 14776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:50.166179 14776 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:50.167054 14776 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:50.176401 14776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aadecf324b843b99ffd8158a35750fa: Generated new TSK 0
I20260927 10:48:50.177024 14776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:48:50.194336 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:50.189543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.9.39.190:36241)
0927 10:48:50.189678 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:50.189683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:50.189855 (+ 172us) server_negotiation.cc:407] Connection header received
0927 10:48:50.190010 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:50.190013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:50.190064 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:50.190150 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0927 10:48:50.190403 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.190968 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.192193 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.192416 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.192489 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:50.192792 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 10:48:50.193945 (+ 1153us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 10:48:50.194197 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40134: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260927 10:48:50.194407 14791 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:50.189450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36241 (local address 127.0.0.1:40134)
0927 10:48:50.189764 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:50.189779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:50.189893 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:50.190179 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:50.190182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:50.190191 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0927 10:48:50.190301 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:50.190308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.191091 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.191095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:50.192077 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:50.192088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.192528 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.192532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:50.192616 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:50.192643 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 10:48:50.194110 (+ 1467us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 10:48:50.194244 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.39.190:36241: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":107,"threads_started":1}
W20260927 10:48:50.194415 14765 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40134: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260927 10:48:50.195133 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14695
E20260927 10:48:50.205549 14792 webserver.cc:533] Webserver: sslize failed:
E20260927 10:48:50.205608 14792 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-27T10:48:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (283 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T10:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:50Z Disabled control of system clock
I20260927 10:48:50.241676 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43221
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:43221
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/testchainca.cert with env {}
W20260927 10:48:50.345814 14796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:50.346110 14796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:50.346213 14796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:50.350189 14796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:50.350268 14796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:50.350297 14796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:50.350339 14796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:50.350376 14796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 10:48:50.350414 14796 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 10:48:50.350443 14796 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 10:48:50.356963 14796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:43221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:50.357933 14796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:50.358973 14796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:50.366830 14801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:50.366856 14804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:50.366878 14802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:50.367048 14796 server_base.cc:1034] running on GCE node
I20260927 10:48:50.367559 14796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:50.368503 14796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:50.369668 14796 hybrid_clock.cc:648] HybridClock initialized: now 1790506130369644 us; error 40 us; skew 500 ppm
I20260927 10:48:50.371423 14796 webserver.cc:460] Webserver started at http://127.9.39.190:44303/ using document root <none> and password file <none>
I20260927 10:48:50.371968 14796 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:50.372025 14796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:50.372267 14796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:50.374256 14796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "1ea7594296984bc6a550ea74c90d4b35"
format_stamp: "Formatted at 2026-09-27 10:48:50 on dist-test-slave-p9q6"
server_key: "bf22b110ba07de2fb2fd2e293fb43c3b"
server_key_iv: "ed1f44ee8eb74e12789ae6675f7d4d7d"
server_key_version: "encryptionkey@0"
I20260927 10:48:50.374913 14796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "1ea7594296984bc6a550ea74c90d4b35"
format_stamp: "Formatted at 2026-09-27 10:48:50 on dist-test-slave-p9q6"
server_key: "bf22b110ba07de2fb2fd2e293fb43c3b"
server_key_iv: "ed1f44ee8eb74e12789ae6675f7d4d7d"
server_key_version: "encryptionkey@0"
I20260927 10:48:50.378460 14796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260927 10:48:50.380777 14810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:50.381953 14796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260927 10:48:50.382088 14796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "1ea7594296984bc6a550ea74c90d4b35"
format_stamp: "Formatted at 2026-09-27 10:48:50 on dist-test-slave-p9q6"
server_key: "bf22b110ba07de2fb2fd2e293fb43c3b"
server_key_iv: "ed1f44ee8eb74e12789ae6675f7d4d7d"
server_key_version: "encryptionkey@0"
I20260927 10:48:50.382195 14796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:50.422959 14796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:50.423857 14796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:50.424065 14796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:50.431397 14796 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:43221
I20260927 10:48:50.431434 14862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:43221 every 8 connection(s)
I20260927 10:48:50.432322 14796 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:50.436234 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:50.436996 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14796
I20260927 10:48:50.437129 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:50.437412 9374 external_mini_cluster.cc:1428] Setting key 95089b3a902df40598d70403159e1611
I20260927 10:48:50.444016 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35: Bootstrap starting.
I20260927 10:48:50.446475 14863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:50.447148 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:50.438830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48140 (local address 127.9.39.190:43221)
0927 10:48:50.439524 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:50.439549 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:50.439595 (+ 46us) server_negotiation.cc:407] Connection header received
0927 10:48:50.440564 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:50.440587 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:50.440999 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:50.441344 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:50.441671 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.442596 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.443785 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.444324 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.444452 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:50.445345 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:50.445349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:50.445360 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:48:50.445737 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:50.445791 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:48:50.445811 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:50.445891 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:50.446059 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 10:48:50.446352 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":134,"threads_started":1}
I20260927 10:48:50.447291 14863 log.cc:824] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:50.449301 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35: No bootstrap required, opened a new log
I20260927 10:48:50.449905 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:48:50.452976 14863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea7594296984bc6a550ea74c90d4b35" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43221 } }
I20260927 10:48:50.453173 14863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:50.453248 14863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea7594296984bc6a550ea74c90d4b35, State: Initialized, Role: FOLLOWER
I20260927 10:48:50.453655 14863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea7594296984bc6a550ea74c90d4b35" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43221 } }
I20260927 10:48:50.453794 14863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:50.453881 14863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:50.453995 14863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:50.454932 14863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea7594296984bc6a550ea74c90d4b35" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43221 } }
I20260927 10:48:50.455281 14863 leader_election.cc:302] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ea7594296984bc6a550ea74c90d4b35; no voters:
I20260927 10:48:50.455619 14863 leader_election.cc:288] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:50.455732 14868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:50.455962 14868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [term 1 LEADER]: Becoming Leader. State: Replica: 1ea7594296984bc6a550ea74c90d4b35, State: Running, Role: LEADER
I20260927 10:48:50.456244 14868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea7594296984bc6a550ea74c90d4b35" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43221 } }
I20260927 10:48:50.456885 14863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:50.457978 14869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ea7594296984bc6a550ea74c90d4b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea7594296984bc6a550ea74c90d4b35" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43221 } } }
I20260927 10:48:50.458132 14869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:50.458425 14870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ea7594296984bc6a550ea74c90d4b35. Latest consensus state: current_term: 1 leader_uuid: "1ea7594296984bc6a550ea74c90d4b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea7594296984bc6a550ea74c90d4b35" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43221 } } }
I20260927 10:48:50.458525 14870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35 [sys.catalog]: This master's current role is: LEADER
W20260927 10:48:50.461485 14883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:48:50.461575 14883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:48:50.461656 14877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:50.462296 14877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:50.467294 14877 catalog_manager.cc:1347] Generated new cluster ID: 052859e647d340d49be29e2a190af2d4
I20260927 10:48:50.467355 14877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:50.476286 14877 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:50.477152 14877 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:50.481873 14877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ea7594296984bc6a550ea74c90d4b35: Generated new TSK 0
I20260927 10:48:50.482457 14877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:48:50.573865 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:50.495604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.9.39.190:43221)
0927 10:48:50.495717 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:50.495723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:50.495898 (+ 175us) server_negotiation.cc:407] Connection header received
0927 10:48:50.496102 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:50.496106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:50.496149 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:50.496250 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0927 10:48:50.496485 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.497036 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.497898 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.498149 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.498392 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:50.498685 (+ 293us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 10:48:50.573370 (+ 74685us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 10:48:50.573710 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260927 10:48:50.573953 14866 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 10:48:50.573976 14892 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:50.495548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43221 (local address 127.0.0.1:48144)
0927 10:48:50.495818 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:50.495829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:50.495961 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:50.496261 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:50.496264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:50.496271 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0927 10:48:50.496374 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:50.496387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.497149 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.497154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:50.497778 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:50.497785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:50.498396 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:50.498400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:50.498490 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:50.498519 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 10:48:50.573606 (+ 75087us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 10:48:50.573798 (+ 192us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.39.190:43221: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":103,"threads_started":1}
I20260927 10:48:50.574728 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14796
2026-09-27T10:48:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (360 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (122 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:50 dist-test-slave-p9q6 krb5kdc[14904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:50 dist-test-slave-p9q6 krb5kdc[14904](info): set up 2 sockets
Sep 27 10:48:50 dist-test-slave-p9q6 krb5kdc[14904](info): commencing operation
krb5kdc: starting...
W20260927 10:48:51.854997 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:51 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:48:51.973886 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40093
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:52.081877 14917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:52.082139 14917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:52.082181 14917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:52.086159 14917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:52.086220 14917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:52.086239 14917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:52.086256 14917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:52.086270 14917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:52.091441 14917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:52.092386 14917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:52.093343 14917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:52.101339 14922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:52.101364 14923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:52.101428 14925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:52.101511 14917 server_base.cc:1034] running on GCE node
I20260927 10:48:52.102239 14917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:52.103305 14917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:52.104501 14917 hybrid_clock.cc:648] HybridClock initialized: now 1790506132104469 us; error 47 us; skew 500 ppm
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.107199 14917 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:48:52.108073 14917 webserver.cc:460] Webserver started at http://127.9.39.190:46495/ using document root <none> and password file <none>
I20260927 10:48:52.108601 14917 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:52.108690 14917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:52.108908 14917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:52.110889 14917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "d2020f0b9c4f4947907178f22a8eedfc"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "4a7dc43bc14e0f924158990617841692"
server_key_iv: "27a8e98dbee770362e6dd1b1d27d55b1"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.111444 14917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "d2020f0b9c4f4947907178f22a8eedfc"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "4a7dc43bc14e0f924158990617841692"
server_key_iv: "27a8e98dbee770362e6dd1b1d27d55b1"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.114964 14917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:52.117100 14932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.118253 14917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:52.118389 14917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "d2020f0b9c4f4947907178f22a8eedfc"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "4a7dc43bc14e0f924158990617841692"
server_key_iv: "27a8e98dbee770362e6dd1b1d27d55b1"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.118495 14917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:52.137172 14917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:52.140110 14917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:52.140336 14917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:52.147359 14917 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:40093
I20260927 10:48:52.147379 14984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:40093 every 8 connection(s)
I20260927 10:48:52.148283 14917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:52.149616 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 14917
I20260927 10:48:52.149708 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:52.149989 9374 external_mini_cluster.cc:1428] Setting key 6057ee11eb6425b86b72b32c3dae3cb8
I20260927 10:48:52.151368 14985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:52.158105 14985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc: Bootstrap starting.
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:52.160511 14985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:52.161252 14985 log.cc:824] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:52.163492 14985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc: No bootstrap required, opened a new log
I20260927 10:48:52.164741 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.151350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.9.39.190:40093)
0927 10:48:52.151761 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.151789 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.151823 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:48:52.152456 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.152473 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.152786 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.153061 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.154156 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.155247 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.156398 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.156869 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.156992 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.159920 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.159925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.159938 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.159966 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.161926 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.162471 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.162476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.162480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.162586 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.162900 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.162902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.162903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.163301 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.163469 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.163753 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.164056 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":170,"threads_started":1}
I20260927 10:48:52.167302 14985 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2020f0b9c4f4947907178f22a8eedfc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40093 } }
I20260927 10:48:52.167539 14985 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:52.167618 14985 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2020f0b9c4f4947907178f22a8eedfc, State: Initialized, Role: FOLLOWER
I20260927 10:48:52.168023 14985 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2020f0b9c4f4947907178f22a8eedfc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40093 } }
I20260927 10:48:52.168169 14985 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:52.168254 14985 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:52.168370 14985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:52.169226 14985 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2020f0b9c4f4947907178f22a8eedfc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40093 } }
I20260927 10:48:52.169664 14985 leader_election.cc:302] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2020f0b9c4f4947907178f22a8eedfc; no voters:
I20260927 10:48:52.170013 14985 leader_election.cc:288] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:52.170078 14990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:52.170257 14990 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [term 1 LEADER]: Becoming Leader. State: Replica: d2020f0b9c4f4947907178f22a8eedfc, State: Running, Role: LEADER
I20260927 10:48:52.170542 14990 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2020f0b9c4f4947907178f22a8eedfc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40093 } }
I20260927 10:48:52.171272 14985 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:52.172240 14992 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2020f0b9c4f4947907178f22a8eedfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2020f0b9c4f4947907178f22a8eedfc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40093 } } }
I20260927 10:48:52.172381 14992 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:52.172786 14999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:52.172806 14991 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2020f0b9c4f4947907178f22a8eedfc. Latest consensus state: current_term: 1 leader_uuid: "d2020f0b9c4f4947907178f22a8eedfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2020f0b9c4f4947907178f22a8eedfc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40093 } } }
I20260927 10:48:52.172996 14991 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:52.175817 14999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:52.181337 14999 catalog_manager.cc:1347] Generated new cluster ID: ba68d83dc3204987a35c070d5d8cfb27
I20260927 10:48:52.181396 14999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:52.197803 14999 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:52.198805 14999 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:52.204813 14999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2020f0b9c4f4947907178f22a8eedfc: Generated new TSK 0
I20260927 10:48:52.205503 14999 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:48:52.243708 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40093
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:52.355744 15011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:52.356077 15011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:52.356163 15011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:52.362042 15011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:52.362120 15011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:52.362238 15011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:52.369226 15011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:52.370628 15011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:52.372207 15011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:52.382118 15016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:52.382094 15019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:52.382161 15011 server_base.cc:1034] running on GCE node
W20260927 10:48:52.382107 15017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:52.382752 15011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:52.383320 15011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:52.384490 15011 hybrid_clock.cc:648] HybridClock initialized: now 1790506132384440 us; error 45 us; skew 500 ppm
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.387187 15011 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:48:52.387985 15011 webserver.cc:460] Webserver started at http://127.9.39.129:33499/ using document root <none> and password file <none>
I20260927 10:48:52.388525 15011 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:52.388615 15011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:52.388833 15011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:52.390830 15011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "480e264c4833490485df63de248a522b"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "625a9834f3f0ed8747e057524365a089"
server_key_iv: "54ebdb4f088912d76746e28e657e9400"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.391419 15011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "480e264c4833490485df63de248a522b"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "625a9834f3f0ed8747e057524365a089"
server_key_iv: "54ebdb4f088912d76746e28e657e9400"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.395017 15011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:52.397291 15026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.398451 15011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:52.398597 15011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "480e264c4833490485df63de248a522b"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "625a9834f3f0ed8747e057524365a089"
server_key_iv: "54ebdb4f088912d76746e28e657e9400"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.398751 15011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:52.420590 15011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:52.423786 15011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:52.423985 15011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:52.424558 15011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:52.425336 15011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:52.425405 15011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.425479 15011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:52.425530 15011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.434098 15011 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:45149
I20260927 10:48:52.434124 15139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:45149 every 8 connection(s)
I20260927 10:48:52.435040 15011 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:52.441812 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15011
I20260927 10:48:52.441931 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:52.442286 9374 external_mini_cluster.cc:1428] Setting key 4870b21ed9dac7ad6dca7d78694f8aa3
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:52.448947 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.436710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:45331 (local address 127.9.39.190:40093)
0927 10:48:52.436870 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.436875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.437694 (+ 819us) server_negotiation.cc:407] Connection header received
0927 10:48:52.439134 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.439138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.439185 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.439260 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.440432 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.440959 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.442153 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.442499 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.442612 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.445126 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.445133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.445138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.445171 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.446767 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.447372 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.447374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.447375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.447481 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.447888 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.447890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.447891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.448055 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.448127 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.448744 (+ 617us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.448835 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 10:48:52.450153 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.437113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40093 (local address 127.9.39.129:45331)
0927 10:48:52.437545 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:52.437585 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:52.438926 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:52.439421 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:52.439427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:52.439828 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:52.440303 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:52.440316 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.441062 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.441064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:52.442025 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:52.442035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.442638 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.442660 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:52.442881 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.443453 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:52.443468 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:52.444950 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:52.446874 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:52.446882 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:52.446899 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:52.447276 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:52.447592 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:52.447598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:52.447603 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:52.447790 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:52.448162 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:52.448171 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:52.448483 (+ 312us) client_negotiation.cc:769] Sending connection context
0927 10:48:52.448716 (+ 233us) client_negotiation.cc:240] Negotiation successful
0927 10:48:52.449084 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260927 10:48:52.451463 15140 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40093
I20260927 10:48:52.451707 15140 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:48:52.452193 15140 heartbeater.cc:511] Master 127.9.39.190:40093 requested a full tablet report, sending...
I20260927 10:48:52.453635 14949 ts_manager.cc:194] Registered new tserver with Master: 480e264c4833490485df63de248a522b (127.9.39.129:45149)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:48:52.454931 14949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:45331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:48:52.470911 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40093
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:52.581460 15145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:52.581677 15145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:52.581717 15145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:52.585491 15145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:52.585537 15145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:52.585588 15145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:52.590606 15145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:52.591534 15145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:52.592429 15145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:52.601651 15150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:52.601687 15151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:52.601681 15153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:52.601770 15145 server_base.cc:1034] running on GCE node
I20260927 10:48:52.602471 15145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:52.603173 15145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:52.604360 15145 hybrid_clock.cc:648] HybridClock initialized: now 1790506132604345 us; error 26 us; skew 500 ppm
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.607076 15145 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:48:52.607870 15145 webserver.cc:460] Webserver started at http://127.9.39.130:37313/ using document root <none> and password file <none>
I20260927 10:48:52.608351 15145 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:52.608405 15145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:52.608566 15145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:52.610455 15145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "a5fed212ee904807844107e35b403e1e"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "7dabf175c4cce2a7d98f56720419bf75"
server_key_iv: "b6df53ebde26dde778db7ef77592d598"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.611014 15145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "a5fed212ee904807844107e35b403e1e"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "7dabf175c4cce2a7d98f56720419bf75"
server_key_iv: "b6df53ebde26dde778db7ef77592d598"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.614425 15145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260927 10:48:52.616688 15160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.617864 15145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:52.617962 15145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "a5fed212ee904807844107e35b403e1e"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "7dabf175c4cce2a7d98f56720419bf75"
server_key_iv: "b6df53ebde26dde778db7ef77592d598"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.618041 15145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:52.650903 15145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:52.654194 15145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:52.654343 15145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:52.654892 15145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:52.655640 15145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:52.655685 15145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.655723 15145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:52.655750 15145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.665475 15145 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:34839
I20260927 10:48:52.665508 15273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:34839 every 8 connection(s)
I20260927 10:48:52.666352 15145 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:52.667523 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15145
I20260927 10:48:52.667624 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:52.667919 9374 external_mini_cluster.cc:1428] Setting key 5781db5feee6c88df3a57c582e33955f
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:48:52.680547 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.668284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:52235 (local address 127.9.39.190:40093)
0927 10:48:52.668415 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.668419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.669378 (+ 959us) server_negotiation.cc:407] Connection header received
0927 10:48:52.670905 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.670908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.670961 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.671067 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.672280 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.672885 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.674206 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.674463 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.674578 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.677195 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.677201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.677204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.677234 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.678534 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.679069 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.679073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.679076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.679157 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.679448 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.679451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.679452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.679651 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.679755 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.680332 (+ 577us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.680423 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 10:48:52.681545 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.668700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40093 (local address 127.9.39.130:52235)
0927 10:48:52.669235 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:52.669290 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:52.670648 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:52.671282 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:52.671290 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:52.671639 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:52.672096 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:52.672109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.673015 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.673018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:52.674087 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:52.674097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.674615 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.674619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:52.674836 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.675424 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:52.675440 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:52.677031 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:52.678638 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:52.678642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:52.678652 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:52.678946 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:52.679235 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:52.679237 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:52.679238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:52.679364 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:52.679766 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:52.679774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:52.680059 (+ 285us) client_negotiation.cc:769] Sending connection context
0927 10:48:52.680262 (+ 203us) client_negotiation.cc:240] Negotiation successful
0927 10:48:52.680609 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":136,"threads_started":1}
I20260927 10:48:52.682732 15274 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40093
I20260927 10:48:52.682972 15274 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:52.683420 15274 heartbeater.cc:511] Master 127.9.39.190:40093 requested a full tablet report, sending...
I20260927 10:48:52.684415 14949 ts_manager.cc:194] Registered new tserver with Master: a5fed212ee904807844107e35b403e1e (127.9.39.130:34839)
I20260927 10:48:52.684985 14949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:52235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:48:52.695643 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40093
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:52.805049 15279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:52.805271 15279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:52.805311 15279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:52.809208 15279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:52.809253 15279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:52.809307 15279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:52.814342 15279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:52.815271 15279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:52.816154 15279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:52.825562 15284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:52.825577 15285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:52.825598 15279 server_base.cc:1034] running on GCE node
W20260927 10:48:52.825567 15287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:52.826277 15279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:52.826979 15279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:52.828162 15279 hybrid_clock.cc:648] HybridClock initialized: now 1790506132828148 us; error 24 us; skew 500 ppm
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.830840 15279 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:48:52.831663 15279 webserver.cc:460] Webserver started at http://127.9.39.131:33973/ using document root <none> and password file <none>
I20260927 10:48:52.832209 15279 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:52.832295 15279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:52.832511 15279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:52.834432 15279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "af8b733cf1a0436880c1f725798e203a"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "8aeff265080d94bc248c630c035abb22"
server_key_iv: "972e1d3114ab3a8b42249a365695f2c7"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.835005 15279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "af8b733cf1a0436880c1f725798e203a"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "8aeff265080d94bc248c630c035abb22"
server_key_iv: "972e1d3114ab3a8b42249a365695f2c7"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.838414 15279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:48:52.840723 15294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.841903 15279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:52.842061 15279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "af8b733cf1a0436880c1f725798e203a"
format_stamp: "Formatted at 2026-09-27 10:48:52 on dist-test-slave-p9q6"
server_key: "8aeff265080d94bc248c630c035abb22"
server_key_iv: "972e1d3114ab3a8b42249a365695f2c7"
server_key_version: "encryptionkey@0"
I20260927 10:48:52.842168 15279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:52.862494 15279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:52.865732 15279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:52.865926 15279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:52.866504 15279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:52.867374 15279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:52.867443 15279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.867504 15279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:52.867535 15279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:52.876361 15279 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:35721
I20260927 10:48:52.876385 15407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:35721 every 8 connection(s)
I20260927 10:48:52.877192 15279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:52.881698 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15279
I20260927 10:48:52.881831 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:52.882093 9374 external_mini_cluster.cc:1428] Setting key a0c5d84f2227be960ea6492629709108
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:48:52.890412 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.878908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:33929 (local address 127.9.39.190:40093)
0927 10:48:52.879042 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.879046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.879797 (+ 751us) server_negotiation.cc:407] Connection header received
0927 10:48:52.881149 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.881153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.881211 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.881319 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.882539 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.883277 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.884475 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.884712 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.884827 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.887053 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.887056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.887059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.887084 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.888407 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.888894 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.888899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.888900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.889001 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.889362 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.889366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.889368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.889564 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.889668 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.890134 (+ 466us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.890249 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 10:48:52.891224 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.879216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40093 (local address 127.9.39.131:33929)
0927 10:48:52.879656 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:52.879697 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:52.880951 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:52.881499 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:52.881505 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:52.881864 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:52.882388 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:52.882402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.883393 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.883396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:52.884359 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:52.884368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.884869 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.884873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:52.885023 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.885484 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:52.885505 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:52.886901 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:52.888534 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:52.888539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:52.888553 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:52.888794 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:52.889133 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:52.889135 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:52.889137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:52.889265 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:52.889664 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:52.889671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:52.889894 (+ 223us) client_negotiation.cc:769] Sending connection context
0927 10:48:52.890070 (+ 176us) client_negotiation.cc:240] Negotiation successful
0927 10:48:52.890372 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":101,"threads_started":1}
I20260927 10:48:52.892235 15408 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40093
I20260927 10:48:52.892441 15408 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:52.892953 15408 heartbeater.cc:511] Master 127.9.39.190:40093 requested a full tablet report, sending...
I20260927 10:48:52.893852 14949 ts_manager.cc:194] Registered new tserver with Master: af8b733cf1a0436880c1f725798e203a (127.9.39.131:35721)
I20260927 10:48:52.894349 14949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:33929
I20260927 10:48:52.895726 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/test-admin.keytab.
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.911479 9374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 10:48:52.914110 15415 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:52.912840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.9.39.129:45149)
0927 10:48:52.913094 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.913108 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.913131 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:48:52.913198 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.913203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.913378 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.913509 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.913931 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57522: BlockingRecv error: recv got EOF from 127.0.0.1:57522 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
W20260927 10:48:52.914325 9374 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc' of type FILE
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.915498 9374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.916409 9374 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:52.925704 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.917215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.9.39.129:45149)
0927 10:48:52.917362 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.917374 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.917468 (+ 94us) server_negotiation.cc:407] Connection header received
0927 10:48:52.917546 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.917548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.917587 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.917661 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.918135 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.918970 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.920001 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.920353 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.920442 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.922790 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.922799 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.922810 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.922831 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.924219 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.924605 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.924610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.924611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.924682 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.924890 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.924893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.924894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.925107 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.925251 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.925390 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.925555 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260927 10:48:52.929144 15415 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:52.928372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.9.39.129:45149)
0927 10:48:52.928496 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.928499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.928524 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:48:52.928624 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.928626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.928656 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.928714 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.929050 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57548: BlockingRecv error: recv got EOF from 127.0.0.1:57548 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260927 10:48:52.929239 9374 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc' of type FILE
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.930094 9374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.930963 9374 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:52.938815 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.931672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57550 (local address 127.9.39.129:45149)
0927 10:48:52.931784 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.931788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.931892 (+ 104us) server_negotiation.cc:407] Connection header received
0927 10:48:52.932015 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.932018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.932059 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.932141 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.932591 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.933081 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.933830 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.934046 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.934152 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.936301 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.936304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.936306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.936324 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.937456 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.937813 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.937815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.937816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.937880 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.938086 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.938088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.938089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.938217 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.938316 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.938488 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.938589 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 10:48:52.941011 15415 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:48:52.940228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.9.39.129:45149)
0927 10:48:52.940332 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.940335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.940476 (+ 141us) server_negotiation.cc:407] Connection header received
0927 10:48:52.940550 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.940551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.940582 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.940641 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.940909 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57562: BlockingRecv error: recv got EOF from 127.0.0.1:57562 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260927 10:48:52.941144 9374 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790506104903010-9374-0/krb5kdc/krb5cc' of type FILE
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.942163 9374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:52.943017 9374 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 10:48:52 dist-test-slave-p9q6 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:48:52.950542 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:52.943672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.9.39.129:45149)
0927 10:48:52.943770 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:52.943774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:52.943877 (+ 103us) server_negotiation.cc:407] Connection header received
0927 10:48:52.943953 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:52.943955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:52.943984 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:52.944045 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:52.944470 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.944922 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.945686 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:52.945878 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:52.945956 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:52.948044 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:52.948047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:52.948049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:52.948068 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:52.949214 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.949572 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.949575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.949576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.949640 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:52.949876 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:52.949879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:52.949880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:52.950033 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:52.950139 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:52.950342 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 10:48:52.950428 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:48:52.951500 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15011
I20260927 10:48:52.957778 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15145
I20260927 10:48:52.963109 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15279
I20260927 10:48:52.969076 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 14917
2026-09-27T10:48:52Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2150 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:48:53 dist-test-slave-p9q6 krb5kdc[15428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:48:53 dist-test-slave-p9q6 krb5kdc[15428](info): set up 2 sockets
Sep 27 10:48:53 dist-test-slave-p9q6 krb5kdc[15428](info): commencing operation
krb5kdc: starting...
W20260927 10:48:54.003089 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:48:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab.
Entry for principal oryx/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab.
I20260927 10:48:54.129498 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:38161
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab
--principal=oryx/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:54.239880 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:54.240154 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:54.240193 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:54.244148 15441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:54.244206 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:54.244225 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:54.244243 15441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:54.244257 15441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:54.249380 15441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab
--principal=oryx/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:54.250347 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:54.251384 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:54.259761 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:54.259794 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:54.260048 15449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:54.260119 15441 server_base.cc:1034] running on GCE node
I20260927 10:48:54.260665 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:54.262197 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:54.263387 15441 hybrid_clock.cc:648] HybridClock initialized: now 1790506134263367 us; error 39 us; skew 500 ppm
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:54.266815 15441 init.cc:377] Logged in from keytab as oryx/127.9.39.190@KRBTEST.COM (short username oryx)
I20260927 10:48:54.267874 15441 webserver.cc:460] Webserver started at http://127.9.39.190:38005/ using document root <none> and password file <none>
I20260927 10:48:54.268451 15441 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:54.268544 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:54.268754 15441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:54.270838 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "83399f1eda404af39814d232d3517772"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ab9eaa16a1d65742072ca19ae057fed1"
server_key_iv: "1405a0a2970a5f4b7ad8461da626a4fb"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.271399 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "83399f1eda404af39814d232d3517772"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ab9eaa16a1d65742072ca19ae057fed1"
server_key_iv: "1405a0a2970a5f4b7ad8461da626a4fb"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.275209 15441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 10:48:54.277614 15456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.278959 15441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:54.279089 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "83399f1eda404af39814d232d3517772"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ab9eaa16a1d65742072ca19ae057fed1"
server_key_iv: "1405a0a2970a5f4b7ad8461da626a4fb"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.279198 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:54.293989 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:54.297250 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:54.297448 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:54.304775 15441 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:38161
I20260927 10:48:54.305084 15508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:38161 every 8 connection(s)
I20260927 10:48:54.305761 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:48:54.308667 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:54.314291 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Bootstrap starting.
I20260927 10:48:54.315761 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15441
I20260927 10:48:54.315912 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:48:54.316169 9374 external_mini_cluster.cc:1428] Setting key 81b4803c8bfc7d682d068bb0ca7dd4fb
I20260927 10:48:54.317024 15509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:54.317956 15509 log.cc:824] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:54.319887 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: No bootstrap required, opened a new log
I20260927 10:48:54.323320 15509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } }
I20260927 10:48:54.323568 15509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:54.323657 15509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83399f1eda404af39814d232d3517772, State: Initialized, Role: FOLLOWER
I20260927 10:48:54.324189 15509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } }
I20260927 10:48:54.324374 15509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:48:54.324446 15509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:48:54.324560 15509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:48:54.325544 15509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } }
I20260927 10:48:54.325948 15509 leader_election.cc:302] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83399f1eda404af39814d232d3517772; no voters:
I20260927 10:48:54.326375 15509 leader_election.cc:288] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:48:54.326474 15514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:54.326727 15514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 LEADER]: Becoming Leader. State: Replica: 83399f1eda404af39814d232d3517772, State: Running, Role: LEADER
I20260927 10:48:54.327102 15514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } }
I20260927 10:48:54.327744 15509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:54.328732 15515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } }
I20260927 10:48:54.328919 15515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:54.329348 15523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:54.330497 15516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83399f1eda404af39814d232d3517772. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } }
I20260927 10:48:54.330602 15516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:54.331368 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:54.317274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.9.39.190:38161)
0927 10:48:54.317769 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:54.317784 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:54.317819 (+ 35us) server_negotiation.cc:407] Connection header received
0927 10:48:54.318521 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:54.318538 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:54.318916 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:54.319255 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:54.320049 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.321016 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.321877 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.322352 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.322524 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:54.325393 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:54.325404 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:54.325418 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:54.325443 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:54.327578 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:54.328052 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:54.328058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:54.328064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:54.328197 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:54.328552 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:54.328555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:54.328558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:54.329117 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:54.329301 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:54.330132 (+ 831us) server_negotiation.cc:299] Negotiation successful
0927 10:48:54.330417 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":151,"threads_started":1}
I20260927 10:48:54.332713 15523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:54.338274 15523 catalog_manager.cc:1347] Generated new cluster ID: 9eecf9d5c0404e83a1640903632c6a4e
I20260927 10:48:54.338354 15523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:54.344441 15523 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:48:54.345260 15523 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:54.353513 15523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Generated new TSK 0
I20260927 10:48:54.354295 15523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab.
Entry for principal oryx/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab.
I20260927 10:48:54.385962 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:38161
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:54.496985 15535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:54.497259 15535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:54.497303 15535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:54.501293 15535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:54.501351 15535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:54.501415 15535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:54.506621 15535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:38161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:54.507659 15535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:54.508649 15535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:54.518219 15540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:54.518196 15543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:54.518332 15535 server_base.cc:1034] running on GCE node
W20260927 10:48:54.518210 15541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:54.518978 15535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:54.519598 15535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:54.520787 15535 hybrid_clock.cc:648] HybridClock initialized: now 1790506134520771 us; error 25 us; skew 500 ppm
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:54.523676 15535 init.cc:377] Logged in from keytab as oryx/127.9.39.129@KRBTEST.COM (short username oryx)
I20260927 10:48:54.524533 15535 webserver.cc:460] Webserver started at http://127.9.39.129:44043/ using document root <none> and password file <none>
I20260927 10:48:54.525043 15535 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:54.525099 15535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:54.525255 15535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:54.527407 15535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "fd60eeebf2084b8baf3617e842cdd7ac"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "3cd096546e60340d2aa699ce522eee2d"
server_key_iv: "463d641837e895e3009be7d08cf88de4"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.527972 15535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "fd60eeebf2084b8baf3617e842cdd7ac"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "3cd096546e60340d2aa699ce522eee2d"
server_key_iv: "463d641837e895e3009be7d08cf88de4"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.531807 15535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:48:54.534103 15550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.535374 15535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:54.535502 15535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "fd60eeebf2084b8baf3617e842cdd7ac"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "3cd096546e60340d2aa699ce522eee2d"
server_key_iv: "463d641837e895e3009be7d08cf88de4"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.535584 15535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:54.549955 15535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:54.552878 15535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:54.553030 15535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:54.553555 15535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:54.554287 15535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:54.554332 15535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.554370 15535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:54.554385 15535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.562881 15535 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:33189
I20260927 10:48:54.562923 15663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:33189 every 8 connection(s)
I20260927 10:48:54.563752 15535 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:54.572221 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15535
I20260927 10:48:54.572364 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:48:54.572669 9374 external_mini_cluster.cc:1428] Setting key 16fabc7e444a1e27008cb3e47804c407
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:48:54.578153 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:54.565506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:45627 (local address 127.9.39.190:38161)
0927 10:48:54.565659 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:54.565662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:54.566661 (+ 999us) server_negotiation.cc:407] Connection header received
0927 10:48:54.567598 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:54.567601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:54.567647 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:54.567721 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:54.568901 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.569468 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.570661 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.570999 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.571092 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:54.573894 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:54.573898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:54.573900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:54.573924 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:54.575560 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:54.576235 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:54.576239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:54.576243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:54.576368 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:54.576758 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:54.576761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:54.576764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:54.577026 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:54.577133 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:54.577740 (+ 607us) server_negotiation.cc:299] Negotiation successful
0927 10:48:54.577899 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:48:54.578955 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:54.565864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:45627)
0927 10:48:54.566534 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:54.566576 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:54.567416 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:54.567877 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:54.567884 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:54.568229 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:54.568777 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:54.568791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.569595 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.569598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:54.570528 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:54.570537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.571163 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.571167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:54.571319 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:54.571970 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:54.571986 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:54.573679 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:54.575679 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:54.575688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:54.575708 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:54.576127 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:54.576469 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:54.576472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:54.576474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:54.576654 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:54.577137 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:54.577144 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:54.577476 (+ 332us) client_negotiation.cc:769] Sending connection context
0927 10:48:54.577652 (+ 176us) client_negotiation.cc:240] Negotiation successful
0927 10:48:54.577925 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":128,"threads_started":1}
I20260927 10:48:54.580265 15664 heartbeater.cc:347] Connected to a master server at 127.9.39.190:38161
I20260927 10:48:54.580559 15664 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:54.581058 15664 heartbeater.cc:511] Master 127.9.39.190:38161 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:48:54.582733 15473 ts_manager.cc:194] Registered new tserver with Master: fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129:33189)
I20260927 10:48:54.584164 15473 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:45627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab.
Entry for principal oryx/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab.
I20260927 10:48:54.602679 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:38161
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:54.715329 15669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:54.715608 15669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:54.715683 15669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:54.719734 15669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:54.719833 15669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:54.719928 15669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:54.725342 15669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:38161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:54.726459 15669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:54.727540 15669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:54.737385 15675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:54.737434 15677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:54.737493 15674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:54.737774 15669 server_base.cc:1034] running on GCE node
I20260927 10:48:54.738162 15669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:54.738907 15669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:54.740089 15669 hybrid_clock.cc:648] HybridClock initialized: now 1790506134740057 us; error 43 us; skew 500 ppm
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:54.743043 15669 init.cc:377] Logged in from keytab as oryx/127.9.39.130@KRBTEST.COM (short username oryx)
I20260927 10:48:54.743978 15669 webserver.cc:460] Webserver started at http://127.9.39.130:38757/ using document root <none> and password file <none>
I20260927 10:48:54.744572 15669 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:54.744663 15669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:54.744894 15669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:54.747000 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "abe31483246640728ab67024a8b71630"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "6be15dd02166680bdc66c1445f03285a"
server_key_iv: "1d0da9c912883177ea12b663e601136b"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.747582 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "abe31483246640728ab67024a8b71630"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "6be15dd02166680bdc66c1445f03285a"
server_key_iv: "1d0da9c912883177ea12b663e601136b"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.751390 15669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260927 10:48:54.753787 15684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.755187 15669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260927 10:48:54.755329 15669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "abe31483246640728ab67024a8b71630"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "6be15dd02166680bdc66c1445f03285a"
server_key_iv: "1d0da9c912883177ea12b663e601136b"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.755443 15669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:54.781581 15669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:54.784902 15669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:54.785106 15669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:54.785699 15669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:54.786513 15669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:54.786558 15669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.786625 15669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:54.786659 15669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.795651 15669 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:41315
I20260927 10:48:54.795676 15797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:41315 every 8 connection(s)
I20260927 10:48:54.796512 15669 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:54.799400 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15669
I20260927 10:48:54.799539 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:48:54.799846 9374 external_mini_cluster.cc:1428] Setting key 41cb77fa0b4c4221f64ceb6e75290270
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:48:54.810755 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:54.798186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:36425 (local address 127.9.39.190:38161)
0927 10:48:54.798313 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:54.798317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:54.799531 (+ 1214us) server_negotiation.cc:407] Connection header received
0927 10:48:54.800646 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:54.800650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:54.800695 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:54.800780 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:54.802279 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.802893 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.804045 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.804287 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.804377 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:54.807232 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:54.807235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:54.807237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:54.807259 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:54.808671 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:54.809202 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:54.809207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:54.809208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:54.809323 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:54.809662 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:54.809667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:54.809668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:54.809848 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:54.809956 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:54.810439 (+ 483us) server_negotiation.cc:299] Negotiation successful
0927 10:48:54.810554 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.131@KRBTEST.COM" created.
I20260927 10:48:54.811518 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:54.798582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:36425)
0927 10:48:54.799354 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:54.799413 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:54.800429 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:54.800913 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:54.800920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:54.801418 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:54.802131 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:54.802152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.803052 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.803055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:54.803936 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:54.803945 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:54.804460 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:54.804467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:54.804694 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:54.805293 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:54.805319 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:54.807062 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:54.808804 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:54.808810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:54.808825 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:54.809097 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:54.809421 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:54.809424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:54.809427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:54.809565 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:54.809957 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:54.809964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:54.810203 (+ 239us) client_negotiation.cc:769] Sending connection context
0927 10:48:54.810390 (+ 187us) client_negotiation.cc:240] Negotiation successful
0927 10:48:54.810694 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":178,"threads_started":1}
I20260927 10:48:54.812705 15798 heartbeater.cc:347] Connected to a master server at 127.9.39.190:38161
I20260927 10:48:54.812958 15798 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:54.813439 15798 heartbeater.cc:511] Master 127.9.39.190:38161 requested a full tablet report, sending...
I20260927 10:48:54.814505 15473 ts_manager.cc:194] Registered new tserver with Master: abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:48:54.815383 15473 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:36425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab.
Entry for principal oryx/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab.
I20260927 10:48:54.827359 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:38161
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:54.937639 15803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:54.937887 15803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:54.937956 15803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:54.942130 15803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:54.942198 15803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:54.942286 15803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:54.947932 15803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:38161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:54.949059 15803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:54.950089 15803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:54.959371 15811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:54.959394 15809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:54.959574 15803 server_base.cc:1034] running on GCE node
W20260927 10:48:54.959461 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:54.959970 15803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:54.960553 15803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:54.961738 15803 hybrid_clock.cc:648] HybridClock initialized: now 1790506134961723 us; error 27 us; skew 500 ppm
Sep 27 10:48:54 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:54.964378 15803 init.cc:377] Logged in from keytab as oryx/127.9.39.131@KRBTEST.COM (short username oryx)
I20260927 10:48:54.965189 15803 webserver.cc:460] Webserver started at http://127.9.39.131:45485/ using document root <none> and password file <none>
I20260927 10:48:54.965729 15803 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:54.965812 15803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:54.966027 15803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:54.968024 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "5efe62e66f6046968b0fc59352b2b122"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ba692b166219681513c7c1bf1fe8670f"
server_key_iv: "65190843e266fa75b2f7df16ea2f18e3"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.968767 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "5efe62e66f6046968b0fc59352b2b122"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ba692b166219681513c7c1bf1fe8670f"
server_key_iv: "65190843e266fa75b2f7df16ea2f18e3"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.972442 15803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:48:54.974747 15818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.975984 15803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:54.976107 15803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "5efe62e66f6046968b0fc59352b2b122"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ba692b166219681513c7c1bf1fe8670f"
server_key_iv: "65190843e266fa75b2f7df16ea2f18e3"
server_key_version: "encryptionkey@0"
I20260927 10:48:54.976212 15803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:54.985582 15803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:54.988340 15803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:54.988536 15803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:54.989063 15803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:54.989840 15803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:54.989905 15803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.989969 15803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:54.990018 15803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:54.998536 15803 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:44767
I20260927 10:48:54.998560 15931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:44767 every 8 connection(s)
I20260927 10:48:54.999442 15803 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:55.003611 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15803
I20260927 10:48:55.003733 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:48:55.004021 9374 external_mini_cluster.cc:1428] Setting key 9043013c4833423f39edeb9535c24d25
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:48:55.013132 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.001433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:35469 (local address 127.9.39.190:38161)
0927 10:48:55.001582 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.001586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.002566 (+ 980us) server_negotiation.cc:407] Connection header received
0927 10:48:55.003512 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.003516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.003577 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.003687 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.005036 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.005760 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.007037 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.007245 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.007338 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.009636 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.009639 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.009641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.009664 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.011238 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.011760 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.011762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.011764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.011839 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.012191 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.012193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.012194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.012362 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.012444 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.012906 (+ 462us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.013013 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 10:48:55.013973 15934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.001799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:35469)
0927 10:48:55.002429 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.002467 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.003288 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.003883 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.003890 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.004284 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.004915 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.004931 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.005934 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.005940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.006927 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.006937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.007398 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.007401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.007534 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.008042 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.008057 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.009492 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.011387 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.011391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.011402 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.011656 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.011959 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.011962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.011964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.012107 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.012460 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.012465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.012689 (+ 224us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.012844 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.013098 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":115,"threads_started":1}
I20260927 10:48:55.015156 15932 heartbeater.cc:347] Connected to a master server at 127.9.39.190:38161
I20260927 10:48:55.015398 15932 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:55.015885 15932 heartbeater.cc:511] Master 127.9.39.190:38161 requested a full tablet report, sending...
I20260927 10:48:55.016990 15473 ts_manager.cc:194] Registered new tserver with Master: 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767)
I20260927 10:48:55.017689 15473 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:35469
I20260927 10:48:55.018100 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.39.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.189.keytab.
Entry for principal oryx/127.9.39.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.189.keytab.
I20260927 10:48:55.054049 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:38161
--webserver_interface=127.9.39.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.189.keytab
--principal=oryx/127.9.39.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:38161,127.9.39.189:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:48:55.161725 15937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:55.162012 15937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:55.162081 15937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:55.166031 15937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:55.166131 15937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:55.166168 15937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:55.166201 15937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:55.166234 15937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:55.171403 15937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:38161,127.9.39.189:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.189.keytab
--principal=oryx/127.9.39.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.39.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
W20260927 10:48:55.171667 15937 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.39.190:38161,127.9.39.189:38161'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 10:48:55.172523 15937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:55.173530 15937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:55.182197 15942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.182340 15945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.182233 15943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.182461 15937 server_base.cc:1034] running on GCE node
I20260927 10:48:55.183171 15937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:55.184252 15937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:55.185429 15937 hybrid_clock.cc:648] HybridClock initialized: now 1790506135185415 us; error 25 us; skew 500 ppm
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:55.188174 15937 init.cc:377] Logged in from keytab as oryx/127.9.39.189@KRBTEST.COM (short username oryx)
I20260927 10:48:55.189184 15937 webserver.cc:460] Webserver started at http://127.9.39.189:37335/ using document root <none> and password file <none>
I20260927 10:48:55.189735 15937 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.189810 15937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.189983 15937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:55.192202 15937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/instance:
uuid: "b5bad59ad42147f898036f6467d37c28"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "58d521b7f64694dbeb013ff523879d0e"
server_key_iv: "7ef54ea1aefe78933fd564a88266f883"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.192801 15937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal/instance:
uuid: "b5bad59ad42147f898036f6467d37c28"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "58d521b7f64694dbeb013ff523879d0e"
server_key_iv: "7ef54ea1aefe78933fd564a88266f883"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.196599 15937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260927 10:48:55.198952 15952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.200222 15937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:48:55.200372 15937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
uuid: "b5bad59ad42147f898036f6467d37c28"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "58d521b7f64694dbeb013ff523879d0e"
server_key_iv: "7ef54ea1aefe78933fd564a88266f883"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.200477 15937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.222496 15937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:55.225412 15937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:55.225651 15937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:55.233052 15937 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.189:38161
I20260927 10:48:55.233083 16004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.189:38161 every 8 connection(s)
I20260927 10:48:55.233964 15937 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
I20260927 10:48:55.236654 16005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:55.240089 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 15937
I20260927 10:48:55.240916 16005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.189@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
I20260927 10:48:55.253487 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.242831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.9.39.190:38161)
0927 10:48:55.242966 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.242971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.243220 (+ 249us) server_negotiation.cc:407] Connection header received
0927 10:48:55.243454 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.243457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.243509 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.243609 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.244509 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.245056 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.246151 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.246364 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.246440 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.249077 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.249080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.249084 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.249116 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.251087 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.251623 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.251628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.251629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.251742 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.252061 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.252066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.252068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.252371 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.252464 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.252980 (+ 516us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.253110 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:48:55.254323 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.241593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.9.39.189:38161)
0927 10:48:55.242026 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.242039 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.242073 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:48:55.242812 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.242829 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.243252 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.243520 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.244232 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.245286 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.246082 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.246544 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.246647 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.249661 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.249668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.249679 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.249702 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.251631 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.252067 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.252072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.252074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.252149 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.252449 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.252452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.252453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.252753 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.252961 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.253174 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.253321 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":285,"thread_start_us":136,"threads_started":1}
I20260927 10:48:55.254323 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.242725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:34618)
0927 10:48:55.243063 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.243109 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.243244 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.243666 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.243670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.244062 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.244378 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.244390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.245174 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.245178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.245987 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.245995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.246495 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.246501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.246644 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.247466 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.247481 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.248915 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.251187 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.251197 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.251206 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.251522 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.251834 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.251836 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.251838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.251963 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.252467 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.252473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.252753 (+ 280us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.252923 (+ 170us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.253276 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":45,"thread_start_us":91,"threads_started":1}
I20260927 10:48:55.255841 16005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.189@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
I20260927 10:48:55.263610 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.256344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:37256)
0927 10:48:55.256456 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.256471 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.256537 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.256764 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.256767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.256904 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.256988 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.256993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.258046 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.258048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.258550 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.258557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.259067 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.259070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.259170 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.259588 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.259598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.260848 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.262312 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.262315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.262316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.262494 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.262825 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.262827 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.262828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.262901 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.263221 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.263223 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.263288 (+ 65us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.263386 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.263481 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":3}
I20260927 10:48:55.263643 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.256405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37256 (local address 127.9.39.189:38161)
0927 10:48:55.256536 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.256542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.256556 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:48:55.256614 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.256617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.256661 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.256734 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.257082 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.257951 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.258663 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.258972 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.259135 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.260976 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.260979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.260981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.261000 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.262218 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.262580 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.262582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.262583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.262649 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.262991 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.262993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.262994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.263146 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.263218 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.263418 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.263529 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":47}
I20260927 10:48:55.266314 16005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Bootstrap starting.
I20260927 10:48:55.268532 16005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:55.269264 16005 log.cc:824] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:55.271104 16005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: No bootstrap required, opened a new log
I20260927 10:48:55.273904 16005 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } }
I20260927 10:48:55.274113 16005 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:55.274189 16005 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bad59ad42147f898036f6467d37c28, State: Initialized, Role: FOLLOWER
I20260927 10:48:55.274641 16005 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } }
I20260927 10:48:55.275126 16011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } } }
I20260927 10:48:55.275239 16011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:48:55.275828 16005 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 10:48:55.280331 16022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:48:55.280418 16022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:48:55.282230 15937 master_runner.cc:428] Detected that this master b5bad59ad42147f898036f6467d37c28 is joining an existing cluster
I20260927 10:48:55.282292 15937 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.39.189:38161
I20260927 10:48:55.293476 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.284474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.9.39.190:38161)
0927 10:48:55.284614 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.284619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.284782 (+ 163us) server_negotiation.cc:407] Connection header received
0927 10:48:55.285011 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.285016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.285067 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.285159 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.285660 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.286185 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.287450 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.287787 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.287963 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.289066 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.289069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.289072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.289096 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.291348 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.291856 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.291860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.291870 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.292207 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.292520 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.292524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.292526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.292741 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.292836 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.293194 (+ 358us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.293311 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 10:48:55.294034 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.284406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:34620)
0927 10:48:55.284683 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.284697 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.284797 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.285186 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.285189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.285411 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.285548 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.285555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.286290 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.286295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.287317 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.287327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.287967 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.287970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.288064 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.288592 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.288608 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.288928 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.291440 (+ 2512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.291443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.291445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.291748 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.292309 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.292312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.292314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.292418 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.292903 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.292907 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.293008 (+ 101us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.293673 (+ 665us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.293850 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":121,"threads_started":1}
I20260927 10:48:55.294304 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.284913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:37260)
0927 10:48:55.286941 (+ 2028us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.286956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.287047 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.287267 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.287270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.287501 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.287617 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.287624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.288590 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.288593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.289416 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.289425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.289980 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.289984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.290090 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.290520 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.290535 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.290978 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.292831 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.292837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.292840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.293021 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.293282 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.293285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.293286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.293370 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.293795 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.293799 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.293883 (+ 84us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.293986 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.294124 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":75,"threads_started":1}
I20260927 10:48:55.294755 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.284913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.9.39.189:38161)
0927 10:48:55.285031 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.285037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.287015 (+ 1978us) server_negotiation.cc:407] Connection header received
0927 10:48:55.287116 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.287121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.287166 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.287247 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.287733 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.288473 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.289549 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.289870 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.289950 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.291115 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.291119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.291122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.291148 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.292736 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.293113 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.293119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.293121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.293190 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.293464 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.293467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.293469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.293696 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.293803 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.294030 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.294618 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":480,"server-negotiator.queue_time_us":50}
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.190@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
I20260927 10:48:55.307570 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.298681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:37272)
0927 10:48:55.299044 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.299084 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.299238 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.299542 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.299549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.299906 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.300007 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.300013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.300708 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.300713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.301344 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.301354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.301788 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.301791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.301918 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.302405 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.302416 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.303927 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.305734 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.305743 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.305749 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.306062 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.306346 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.306350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.306352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.306442 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.306946 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.306952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.307072 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.307208 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.307408 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260927 10:48:55.308063 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.298744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.9.39.189:38161)
0927 10:48:55.298831 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.298836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.299159 (+ 323us) server_negotiation.cc:407] Connection header received
0927 10:48:55.299370 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.299373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.299426 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.299509 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.300119 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.300600 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.301461 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.301682 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.301788 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.304045 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.304048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.304050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.304070 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.305620 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.306162 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.306169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.306171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.306251 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.306540 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.306543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.306544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.306799 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.306890 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.307813 (+ 923us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.307920 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 10:48:55.310418 15473 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.39.189:38161: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b5bad59ad42147f898036f6467d37c28"
member_type: NON_VOTER
last_known_addr {
host: "127.9.39.189"
port: 38161
}
attrs {
promote: true
}
}
dest_uuid: "83399f1eda404af39814d232d3517772"
cas_config_opid_index: -1
I20260927 10:48:55.311179 15473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } }
I20260927 10:48:55.312873 16032 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b5bad59ad42147f898036f6467d37c28 (127.9.39.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.313496 15979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:55.314523 16032 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.39.189:38161
I20260927 10:48:55.314852 15979 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Refusing update from remote peer 83399f1eda404af39814d232d3517772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 10:48:55.315053 15937 master.cc:537] Master@127.9.39.189:38161 shutting down...
I20260927 10:48:55.315039 16032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.315122 16032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:55.315793 16032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 10:48:55.317998 15460 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260927 10:48:55.318367 15504 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.9.39.190:38161
I20260927 10:48:55.319514 15937 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:55.319756 15937 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:55.319833 15937 tablet_replica.cc:330] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: stopping tablet replica
W20260927 10:48:55.320531 15460 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 -> Peer b5bad59ad42147f898036f6467d37c28 (127.9.39.189:38161): Couldn't send request to peer b5bad59ad42147f898036f6467d37c28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
WARNING: no policy specified for oryx/127.9.39.188@KRBTEST.COM; defaulting to no policy
I20260927 10:48:55.324632 15937 master.cc:552] Master@127.9.39.189:38161 shutdown complete.
I20260927 10:48:55.324723 15937 master_runner.cc:305] Clearing existing system tablet
I20260927 10:48:55.324996 15937 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.325052 15937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.326239 15937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.188@KRBTEST.COM" created.
I20260927 10:48:55.327455 16037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.327669 15937 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.327816 15937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 10:48:55.327867 15937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
uuid: "b5bad59ad42147f898036f6467d37c28"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "58d521b7f64694dbeb013ff523879d0e"
server_key_iv: "7ef54ea1aefe78933fd564a88266f883"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.329921 15937 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:55.332875 15937 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 10:48:55.332978 15937 log.cc:1196] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 10:48:55.333323 15937 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Deleting consensus metadata
I20260927 10:48:55.333544 15937 master_runner.cc:315] Copying system tablet from 127.9.39.190:38161
I20260927 10:48:55.334856 15937 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: tablet copy: Beginning tablet copy session from remote peer at address 127.9.39.190:38161
I20260927 10:48:55.343942 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.335447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:34630)
0927 10:48:55.335732 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.335746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.335854 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.336140 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.336143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.336347 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.336466 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.336473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.337319 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.337323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.338269 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.338278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.338884 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.338889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.338980 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.340008 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.340022 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.340419 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.342147 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.342152 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.342154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.342363 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.342780 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.342785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.342791 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.342871 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.343340 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.343355 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.343461 (+ 106us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.343569 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.343734 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":78,"threads_started":1}
I20260927 10:48:55.343935 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.335574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.9.39.190:38161)
0927 10:48:55.335714 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.335721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.335837 (+ 116us) server_negotiation.cc:407] Connection header received
0927 10:48:55.335952 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.335957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.336009 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.336111 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.336602 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.337214 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.338419 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.338751 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.338890 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.340594 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.340596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.340598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.340621 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.342044 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.342472 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.342476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.342478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.342605 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.342969 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.342974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.342975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.343236 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.343340 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.343654 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.343772 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 10:48:55.344472 15493 tablet_copy_service.cc:140] P 83399f1eda404af39814d232d3517772: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b5bad59ad42147f898036f6467d37c28 ({username='oryx', principal='oryx/127.9.39.189@KRBTEST.COM'} at 127.0.0.1:34630)
I20260927 10:48:55.344643 15493 tablet_copy_service.cc:161] P 83399f1eda404af39814d232d3517772: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b5bad59ad42147f898036f6467d37c28 at {username='oryx', principal='oryx/127.9.39.189@KRBTEST.COM'} at 127.0.0.1:34630: session id = b5bad59ad42147f898036f6467d37c28-00000000000000000000000000000000
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.188.keytab.
Entry for principal oryx/127.9.39.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.188.keytab.
I20260927 10:48:55.346196 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:40695
--webserver_interface=127.9.39.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.188.keytab
--principal=oryx/127.9.39.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:48:55.346513 15493 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 10:48:55.348104 15937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 10:48:55.351074 15937 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: tablet copy: Starting download of 0 data blocks...
I20260927 10:48:55.351289 15937 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: tablet copy: Starting download of 1 WAL segments...
I20260927 10:48:55.353147 15937 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 10:48:55.355360 15493 tablet_copy_service.cc:342] P 83399f1eda404af39814d232d3517772: Request end of tablet copy session b5bad59ad42147f898036f6467d37c28-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.39.189@KRBTEST.COM'} at 127.0.0.1:34630
I20260927 10:48:55.355489 15493 tablet_copy_service.cc:433] P 83399f1eda404af39814d232d3517772: ending tablet copy session b5bad59ad42147f898036f6467d37c28-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b5bad59ad42147f898036f6467d37c28
I20260927 10:48:55.364002 15937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:55.365701 16046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.365744 16047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.365950 15937 server_base.cc:1034] running on GCE node
W20260927 10:48:55.366391 16049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.366609 15937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:55.366914 15937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:55.368055 15937 hybrid_clock.cc:648] HybridClock initialized: now 1790506135368039 us; error 32 us; skew 500 ppm
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:55.369622 15937 init.cc:377] Logged in from keytab as oryx/127.9.39.189@KRBTEST.COM (short username oryx)
I20260927 10:48:55.370042 15937 webserver.cc:460] Webserver started at http://127.9.39.189:33295/ using document root <none> and password file <none>
I20260927 10:48:55.370224 15937 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.370265 15937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.371562 15937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:55.372534 16056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.372752 15937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:55.372816 15937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
uuid: "b5bad59ad42147f898036f6467d37c28"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "58d521b7f64694dbeb013ff523879d0e"
server_key_iv: "7ef54ea1aefe78933fd564a88266f883"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.372893 15937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.402133 15937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:55.404367 15937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:55.409022 15937 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.189:38161
I20260927 10:48:55.409058 16108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.189:38161 every 8 connection(s)
I20260927 10:48:55.409686 15937 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
I20260927 10:48:55.411511 16109 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:48:55.412523 16109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Bootstrap starting.
I20260927 10:48:55.422894 16109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:48:55.423226 16109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Bootstrap complete.
I20260927 10:48:55.423825 16109 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } }
I20260927 10:48:55.423982 16109 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b5bad59ad42147f898036f6467d37c28, State: Initialized, Role: LEARNER
I20260927 10:48:55.424139 16109 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } }
I20260927 10:48:55.424525 16112 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.424640 16109 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:55.424724 16112 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: LEARNER
I20260927 10:48:55.427917 16123 catalog_manager.cc:1259] Loaded cluster ID: 9eecf9d5c0404e83a1640903632c6a4e
I20260927 10:48:55.428030 16123 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: loading cluster ID for follower catalog manager: success
I20260927 10:48:55.430313 16123 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: acquiring CA information for follower catalog manager: success
I20260927 10:48:55.431731 16123 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 10:48:55.477074 16045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:55.477411 16045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:55.477453 16045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:55.481516 16045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:48:55.481570 16045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:55.481590 16045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:55.481606 16045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:48:55.481621 16045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:48:55.486937 16045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.188.keytab
--principal=oryx/127.9.39.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:40695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.39.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:55.487938 16045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:55.488884 16045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:55.498180 16128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.498169 16129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.498263 16045 server_base.cc:1034] running on GCE node
W20260927 10:48:55.498174 16131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.499089 16045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:55.500162 16045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:55.501356 16045 hybrid_clock.cc:648] HybridClock initialized: now 1790506135501308 us; error 54 us; skew 500 ppm
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:55.504061 16045 init.cc:377] Logged in from keytab as oryx/127.9.39.188@KRBTEST.COM (short username oryx)
I20260927 10:48:55.504947 16045 webserver.cc:460] Webserver started at http://127.9.39.188:40415/ using document root <none> and password file <none>
I20260927 10:48:55.505592 16045 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.505685 16045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.505923 16045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:48:55.507977 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/instance:
uuid: "d4a39675396e464a9b7fd6239123abcd"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "4ff86800978d74e0bf810607f132d013"
server_key_iv: "d60e87941c1ff87648eb6aedb01f96be"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.508539 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal/instance:
uuid: "d4a39675396e464a9b7fd6239123abcd"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "4ff86800978d74e0bf810607f132d013"
server_key_iv: "d60e87941c1ff87648eb6aedb01f96be"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.512154 16045 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:55.514573 16138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.515775 16045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:55.515918 16045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
uuid: "d4a39675396e464a9b7fd6239123abcd"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "4ff86800978d74e0bf810607f132d013"
server_key_iv: "d60e87941c1ff87648eb6aedb01f96be"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.516042 16045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.534895 16045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:55.537772 16045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:55.537977 16045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:55.545495 16045 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.188:40695
I20260927 10:48:55.545802 16190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.188:40695 every 8 connection(s)
I20260927 10:48:55.546530 16045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
I20260927 10:48:55.549149 16191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:55.553072 16191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:48:55.553560 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 16045
W20260927 10:48:55.555619 15508 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.9.39.190:38161: Invalid argument: socket address is ambiguous
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.188@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.39.188@KRBTEST.COM
I20260927 10:48:55.567344 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.555816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.9.39.190:38161)
0927 10:48:55.555935 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.555940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.556017 (+ 77us) server_negotiation.cc:407] Connection header received
0927 10:48:55.556543 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.556547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.556607 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.556683 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.557804 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.558637 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.560146 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.560493 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.560619 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.563499 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.563503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.563509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.563540 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.565107 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.565688 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.565692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.565694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.565770 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.566155 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.566162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.566166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.566426 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.566527 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.567058 (+ 531us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.567180 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 10:48:55.568074 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.555578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50174)
0927 10:48:55.555885 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.555930 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.556322 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.556754 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.556762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.557311 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.557675 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.557688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.558795 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.558798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.560033 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.560045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.560677 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.560681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.560783 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.561363 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.561385 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.563287 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.565233 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.565241 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.565253 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.565580 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.565878 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.565881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.565883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.566049 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.566535 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.566542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.566830 (+ 288us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.567014 (+ 184us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.567305 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260927 10:48:55.568077 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.555031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.9.39.188:40695)
0927 10:48:55.555410 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.555423 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.555453 (+ 30us) server_negotiation.cc:407] Connection header received
0927 10:48:55.556269 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.556286 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.556495 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.556674 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.557616 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.558915 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.560041 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.560566 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.560707 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.564079 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.564088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.564103 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.564123 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.565723 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.566225 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.566231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.566232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.566303 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.566602 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.566606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.566607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.566917 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.567641 (+ 724us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.567751 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.567879 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":117,"threads_started":1}
I20260927 10:48:55.569442 16191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.188@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
I20260927 10:48:55.577476 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.569891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:40910)
0927 10:48:55.570293 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.570306 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.570373 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.570624 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.570627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.570831 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.570933 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.570938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.571797 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.571799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.572320 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.572356 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.572733 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.572735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.572803 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.573264 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.573280 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.574518 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.576047 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.576050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.576051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.576219 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.576475 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.576479 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.576481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.576576 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.576937 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.576940 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.577028 (+ 88us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.577109 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.577208 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304}
I20260927 10:48:55.577507 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.570028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.9.39.189:38161)
0927 10:48:55.570332 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.570335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.570346 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:48:55.570452 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.570454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.570491 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.570562 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.571042 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.571706 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.572442 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.572647 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.572719 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.574637 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.574640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.574642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.574663 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.575949 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.576300 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.576303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.576304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.576382 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.576682 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.576685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.576686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.576860 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.576934 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.577162 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.577339 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":103,"threads_started":1}
I20260927 10:48:55.578151 16191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.188@KRBTEST.COM for oryx/127.9.39.188@KRBTEST.COM
I20260927 10:48:55.585726 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.578639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.0.0.1:39418)
0927 10:48:55.578767 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.578780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.578870 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.579090 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.579092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.579233 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.579285 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.579290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.580321 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.580329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.580842 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.580850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.581293 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.581296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.581397 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.581745 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.581754 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.582949 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.584544 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.584546 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.584547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.584720 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.584948 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.584950 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.584951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.585024 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.585321 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.585322 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.585411 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.585489 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.585572 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":20}
I20260927 10:48:55.585733 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.578762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.9.39.188:40695)
0927 10:48:55.578857 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.578861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.578872 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:48:55.578949 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.578951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.578982 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.579067 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.579387 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.579850 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.580951 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.581189 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.581296 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.583074 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.583077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.583079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.583098 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.584456 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.584797 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.584800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.584801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.584877 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.585093 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.585094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.585096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.585248 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.585316 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.585541 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.585618 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":43}
I20260927 10:48:55.587066 15664 heartbeater.cc:503] Master 127.9.39.190:38161 was elected leader, sending a full tablet report...
I20260927 10:48:55.588876 16191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Bootstrap starting.
I20260927 10:48:55.591097 16191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:55.591790 16191 log.cc:824] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:55.593552 16191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: No bootstrap required, opened a new log
I20260927 10:48:55.596462 16191 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } }
I20260927 10:48:55.596645 16191 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:55.596676 16191 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4a39675396e464a9b7fd6239123abcd, State: Initialized, Role: FOLLOWER
I20260927 10:48:55.597091 16191 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } }
I20260927 10:48:55.597536 16197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } } }
I20260927 10:48:55.597646 16197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:48:55.598209 16191 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: configured and running, proceeding with master startup.
W20260927 10:48:55.602906 16208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:48:55.603030 16208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:48:55.606804 16045 master_runner.cc:428] Detected that this master d4a39675396e464a9b7fd6239123abcd is joining an existing cluster
I20260927 10:48:55.606912 16045 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.39.188:40695
I20260927 10:48:55.618486 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.609953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.9.39.190:38161)
0927 10:48:55.610091 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.610097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.610554 (+ 457us) server_negotiation.cc:407] Connection header received
0927 10:48:55.610789 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.610793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.610859 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.610973 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.611610 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.612425 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.613706 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.614043 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614121 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.615644 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.615647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.615649 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.615672 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.616996 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.617449 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.617453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.617455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.617527 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.617788 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.617790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.617791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.617958 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.618032 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.618277 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.618365 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 10:48:55.618566 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.609866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50186)
0927 10:48:55.610455 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.610471 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.610640 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.610991 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.610994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.611232 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.611465 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.611473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.612586 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.612590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.613567 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.613577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.614289 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.614390 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.615013 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.615028 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.615495 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.617096 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.617100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.617102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.617356 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.617609 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.617613 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.617615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.617705 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.618042 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.618045 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.618137 (+ 92us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.618241 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.618366 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":32,"thread_start_us":117,"threads_started":1}
I20260927 10:48:55.621423 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15535
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506134, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.190@KRBTEST.COM for oryx/127.9.39.188@KRBTEST.COM
I20260927 10:48:55.629317 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:33189
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:38161
--webserver_port=44043
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:48:55.643476 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.622448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.0.0.1:39444)
0927 10:48:55.622573 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.622590 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.622670 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.624395 (+ 1725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.624398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.624575 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.624658 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.624663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.625441 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.625445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.626187 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.626196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.626667 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.626671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.626778 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.627255 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.627276 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.628598 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.641736 (+ 13138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.641739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.641742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.642013 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.642326 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.642330 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.642331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.642421 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.642928 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.642931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.643033 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.643118 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.643291 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 10:48:55.643486 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.610603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:40916)
0927 10:48:55.611249 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.611261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.611385 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.611770 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.611775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.611968 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.612067 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.612073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.613131 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.613136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.613958 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.613968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.614657 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.614794 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.639268 (+ 24474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.639291 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.639854 (+ 563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.641894 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.641901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.641905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.642143 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.642434 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.642437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.642439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.642527 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.642995 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.642998 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.643100 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.643194 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.643318 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":123,"threads_started":1}
I20260927 10:48:55.644022 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.610432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.9.39.189:38161)
0927 10:48:55.610580 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.610585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.611325 (+ 740us) server_negotiation.cc:407] Connection header received
0927 10:48:55.611516 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.611520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.611578 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.611632 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.612199 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.612995 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614072 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.614427 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614520 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.640091 (+ 25571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.640098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.640104 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.640147 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.641775 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.642248 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.642254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.642259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.642338 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.642633 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.642636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.642637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.642855 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.642954 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.643707 (+ 753us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.643857 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 10:48:55.646150 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.623926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.9.39.188:40695)
0927 10:48:55.624174 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.624180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.624191 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:48:55.624242 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.624244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.624281 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.624379 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.624767 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.625335 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.626297 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.626555 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.626739 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.628754 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.628758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.628760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.628787 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.641585 (+ 12798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.642136 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.642139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.642141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.642223 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.642543 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.642548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.642553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.642810 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.645292 (+ 2482us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.645645 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.645805 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":113,"threads_started":1}
I20260927 10:48:55.647279 15473 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.39.188:40695: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d4a39675396e464a9b7fd6239123abcd"
member_type: NON_VOTER
last_known_addr {
host: "127.9.39.188"
port: 40695
}
attrs {
promote: true
}
}
dest_uuid: "83399f1eda404af39814d232d3517772"
cas_config_opid_index: 5
I20260927 10:48:55.648018 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.609909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.9.39.188:40695)
0927 10:48:55.610046 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.610051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.611080 (+ 1029us) server_negotiation.cc:407] Connection header received
0927 10:48:55.611115 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.611118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.611154 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.611221 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.611787 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.612517 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.613993 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.614319 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614387 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.644534 (+ 30147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.644537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.644540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.644560 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.646361 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.646846 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.646849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.646851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.646929 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.647200 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.647203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.647204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.647402 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.647495 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.647788 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.647887 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":58}
I20260927 10:48:55.647992 15473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } }
I20260927 10:48:55.648432 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.610137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.0.0.1:39434)
0927 10:48:55.610934 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.610949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.611029 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.611322 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.611324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.611554 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.611682 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.611689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.612622 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.612624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.613864 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.613873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.614560 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.614563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.614659 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.640459 (+ 25800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.640479 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.644405 (+ 3926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.646481 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.646484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.646486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.646729 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.647026 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.647031 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.647033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.647113 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.647539 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.647541 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.647640 (+ 99us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.648160 (+ 520us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.648280 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":53,"thread_start_us":101,"threads_started":1}
I20260927 10:48:55.649170 16033 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d4a39675396e464a9b7fd6239123abcd (127.9.39.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.652099 16165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:55.653502 16165 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Refusing update from remote peer 83399f1eda404af39814d232d3517772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 10:48:55.654049 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.649283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:40922)
0927 10:48:55.649530 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.649546 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.649623 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.650159 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.650163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.650185 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:55.650265 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.650270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.651323 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.651326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.652495 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.652505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.653660 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.653664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.653744 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.653787 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.653862 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.653935 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260927 10:48:55.654135 16032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:55.654476 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.649755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.9.39.189:38161)
0927 10:48:55.649855 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.649858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.649869 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:48:55.649906 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.649908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.649961 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.650047 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:55.650384 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.651210 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.652639 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.653309 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.653408 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.653583 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.654283 (+ 700us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.654357 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:48:55.655112 16083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 LEARNER]: Refusing update from remote peer 83399f1eda404af39814d232d3517772: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 10:48:55.655634 16032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 10:48:55.656032 16033 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.39.188:40695
I20260927 10:48:55.656199 16033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.656299 16033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:55.656436 16045 master.cc:537] Master@127.9.39.188:40695 shutting down...
I20260927 10:48:55.657713 16083 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d4a39675396e464a9b7fd6239123abcd (127.9.39.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.661705 16165 raft_consensus.cc:858] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } }
I20260927 10:48:55.663280 16112 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83399f1eda404af39814d232d3517772. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.663419 16112 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: LEARNER
I20260927 10:48:55.663518 16112 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.663580 16112 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: LEARNER
W20260927 10:48:55.663730 16165 raft_consensus.cc:1565] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } }
I20260927 10:48:55.665335 16165 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b5bad59ad42147f898036f6467d37c28 (127.9.39.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.667309 16221 mvcc.cc:205] Tried to move back new op lower bound from 7333913126307241984 to 7333913126204727296. Current Snapshot: MvccSnapshot[applied={T|T < 7333913126240210944}]
I20260927 10:48:55.667335 16197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83399f1eda404af39814d232d3517772. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.667433 16197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 10:48:55.667531 16197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.667603 16197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 10:48:55.667670 16197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } }
I20260927 10:48:55.667726 16197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 10:48:55.668200 16045 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260927 10:48:55.668406 16045 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260927 10:48:55.668486 16045 tablet_replica.cc:330] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: stopping tablet replica
I20260927 10:48:55.668537 16032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: NON_VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: true } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.668658 16032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
W20260927 10:48:55.670215 15457 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 -> Peer d4a39675396e464a9b7fd6239123abcd (127.9.39.188:40695): Couldn't send request to peer d4a39675396e464a9b7fd6239123abcd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 10:48:55.676352 16033 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: attempting to promote NON_VOTER b5bad59ad42147f898036f6467d37c28 to VOTER
I20260927 10:48:55.676851 16033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } }
I20260927 10:48:55.678488 16083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 LEARNER]: Refusing update from remote peer 83399f1eda404af39814d232d3517772: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260927 10:48:55.678833 15457 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 -> Peer d4a39675396e464a9b7fd6239123abcd (127.9.39.188:40695): Couldn't send request to peer d4a39675396e464a9b7fd6239123abcd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 10:48:55.678925 16033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 10:48:55.680692 16032 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b5bad59ad42147f898036f6467d37c28 (127.9.39.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.682792 16033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.682878 16033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:55.682684 16083 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b5bad59ad42147f898036f6467d37c28 (127.9.39.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.682624 16032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.683142 16032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:55.684119 16222 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.684250 16222 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:48:55.688738 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.688385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.0.0.1:39456)
0927 10:48:55.688610 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.688664 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:40695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260927 10:48:55.693049 16045 master.cc:552] Master@127.9.39.188:40695 shutdown complete.
I20260927 10:48:55.693123 16045 master_runner.cc:305] Clearing existing system tablet
I20260927 10:48:55.693360 16045 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.693432 16045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.694521 16045 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:55.695425 16231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.695647 16045 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.695798 16045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 10:48:55.695854 16045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
uuid: "d4a39675396e464a9b7fd6239123abcd"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "4ff86800978d74e0bf810607f132d013"
server_key_iv: "d60e87941c1ff87648eb6aedb01f96be"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.697656 16045 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:55.699982 16045 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 10:48:55.700091 16045 log.cc:1196] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 10:48:55.700399 16045 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Deleting consensus metadata
I20260927 10:48:55.700583 16045 master_runner.cc:315] Copying system tablet from 127.9.39.190:38161
I20260927 10:48:55.701691 16045 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: tablet copy: Beginning tablet copy session from remote peer at address 127.9.39.190:38161
I20260927 10:48:55.709823 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.702239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50192)
0927 10:48:55.702590 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.702602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.702754 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.703047 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.703051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.703262 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.703374 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.703380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.704158 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.704163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.704758 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.704766 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.705245 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.705249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.705349 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.705901 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.705913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.706240 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.708307 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.708309 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.708311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.708496 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.708780 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.708784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.708786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.708875 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.709245 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.709248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.709359 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.709469 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.709617 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":95,"threads_started":1}
I20260927 10:48:55.709815 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.702319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.9.39.190:38161)
0927 10:48:55.702456 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.702461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.702688 (+ 227us) server_negotiation.cc:407] Connection header received
0927 10:48:55.702880 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.702883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.702940 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.703030 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.703513 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.704044 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.704896 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.705129 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.705239 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.706390 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.706393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.706396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.706421 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.708178 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.708598 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.708602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.708604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.708689 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.708964 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.708969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.708970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.709151 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.709256 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.709532 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.709643 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 10:48:55.710659 15493 tablet_copy_service.cc:140] P 83399f1eda404af39814d232d3517772: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d4a39675396e464a9b7fd6239123abcd ({username='oryx', principal='oryx/127.9.39.188@KRBTEST.COM'} at 127.0.0.1:50192)
I20260927 10:48:55.710813 15493 tablet_copy_service.cc:161] P 83399f1eda404af39814d232d3517772: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d4a39675396e464a9b7fd6239123abcd at {username='oryx', principal='oryx/127.9.39.188@KRBTEST.COM'} at 127.0.0.1:50192: session id = d4a39675396e464a9b7fd6239123abcd-00000000000000000000000000000000
I20260927 10:48:55.711650 15493 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 10:48:55.712781 16045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 10:48:55.714560 16045 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: tablet copy: Starting download of 0 data blocks...
I20260927 10:48:55.714772 16045 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: tablet copy: Starting download of 1 WAL segments...
I20260927 10:48:55.716109 16045 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 10:48:55.717655 15493 tablet_copy_service.cc:342] P 83399f1eda404af39814d232d3517772: Request end of tablet copy session d4a39675396e464a9b7fd6239123abcd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.39.188@KRBTEST.COM'} at 127.0.0.1:50192
I20260927 10:48:55.717761 15493 tablet_copy_service.cc:433] P 83399f1eda404af39814d232d3517772: ending tablet copy session d4a39675396e464a9b7fd6239123abcd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d4a39675396e464a9b7fd6239123abcd
I20260927 10:48:55.724483 16045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:55.726204 16238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.726271 16241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.726637 16239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.726804 16045 server_base.cc:1034] running on GCE node
I20260927 10:48:55.726998 16045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:55.727252 16045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:55.728380 16045 hybrid_clock.cc:648] HybridClock initialized: now 1790506135728334 us; error 71 us; skew 500 ppm
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:55.730001 16045 init.cc:377] Logged in from keytab as oryx/127.9.39.188@KRBTEST.COM (short username oryx)
I20260927 10:48:55.730507 16045 webserver.cc:460] Webserver started at http://127.9.39.188:40841/ using document root <none> and password file <none>
I20260927 10:48:55.730743 16045 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.730791 16045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.731884 16045 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:55.732812 16248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.733042 16045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 10:48:55.733131 16045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
uuid: "d4a39675396e464a9b7fd6239123abcd"
format_stamp: "Formatted at 2026-09-27 10:48:55 on dist-test-slave-p9q6"
server_key: "4ff86800978d74e0bf810607f132d013"
server_key_iv: "d60e87941c1ff87648eb6aedb01f96be"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.733217 16045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.748302 16045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:55.750520 16045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:55.755438 16045 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.188:40695
I20260927 10:48:55.755452 16300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.188:40695 every 8 connection(s)
I20260927 10:48:55.755986 16045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
I20260927 10:48:55.757573 16301 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:48:55.758535 16301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Bootstrap starting.
I20260927 10:48:55.765311 16301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:48:55.765609 16301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: Bootstrap complete.
I20260927 10:48:55.766135 16301 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } }
I20260927 10:48:55.766264 16301 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d4a39675396e464a9b7fd6239123abcd, State: Initialized, Role: LEARNER
I20260927 10:48:55.766393 16301 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } }
I20260927 10:48:55.766750 16304 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:55.766939 16304 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: LEARNER
I20260927 10:48:55.766942 16301 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:48:55.769752 16315 catalog_manager.cc:1259] Loaded cluster ID: 9eecf9d5c0404e83a1640903632c6a4e
I20260927 10:48:55.769845 16315 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: loading cluster ID for follower catalog manager: success
I20260927 10:48:55.771859 16315 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: acquiring CA information for follower catalog manager: success
I20260927 10:48:55.773128 16315 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 10:48:55.789167 16218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:55.789572 16218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:55.789647 16218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:55.795598 16218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:55.795667 16218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:55.795753 16218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:48:55.803503 16218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:33189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=44043
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:55.804811 16218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:55.806034 16218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:48:55.817408 15798 heartbeater.cc:503] Master 127.9.39.190:38161 was elected leader, sending a full tablet report...
W20260927 10:48:55.817374 16321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.817521 16323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:55.817368 16320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:55.818084 16218 server_base.cc:1034] running on GCE node
I20260927 10:48:55.818563 16218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:55.819410 16218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:55.820626 16218 hybrid_clock.cc:648] HybridClock initialized: now 1790506135820583 us; error 49 us; skew 500 ppm
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:55.823901 16218 init.cc:377] Logged in from keytab as oryx/127.9.39.129@KRBTEST.COM (short username oryx)
I20260927 10:48:55.824790 16218 webserver.cc:460] Webserver started at http://127.9.39.129:44043/ using document root <none> and password file <none>
I20260927 10:48:55.825498 16218 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:55.825577 16218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:55.829674 16218 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:48:55.831707 16330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.832881 16218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:55.833019 16218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "fd60eeebf2084b8baf3617e842cdd7ac"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "3cd096546e60340d2aa699ce522eee2d"
server_key_iv: "463d641837e895e3009be7d08cf88de4"
server_key_version: "encryptionkey@0"
I20260927 10:48:55.833602 16218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:55.855568 16218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:55.858605 16218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:55.858834 16218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:55.859442 16218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:55.860297 16218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:55.860368 16218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.860433 16218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:55.860481 16218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:55.869426 16218 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:33189
I20260927 10:48:55.869500 16443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:33189 every 8 connection(s)
I20260927 10:48:55.870410 16218 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:48:55.880633 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 16218
I20260927 10:48:55.880846 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15669
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
Sep 27 10:48:55 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506135, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.188@KRBTEST.COM
I20260927 10:48:55.888105 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.872197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:33427 (local address 127.9.39.189:38161)
0927 10:48:55.872396 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.872403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.873720 (+ 1317us) server_negotiation.cc:407] Connection header received
0927 10:48:55.874952 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.874955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.875016 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.875100 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.876604 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.877399 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879051 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.879392 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879479 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.883516 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.883520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.883523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.883550 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.885567 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.886227 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.886233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.886238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.886335 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.886762 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.886765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.886767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.887032 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.887113 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.887803 (+ 690us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.887935 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:48:55.889159 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.874000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:34757)
0927 10:48:55.874267 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.874281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.874746 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.875337 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.875352 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.875873 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.876464 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.876473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.877566 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.877569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.878941 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.878947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.879444 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.879651 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.880271 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.880294 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.883320 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.886274 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.886276 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.886277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.886527 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.886851 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.886855 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.886856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.886930 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.887367 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.887369 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.887525 (+ 156us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.887731 (+ 206us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.888063 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":270,"spinlock_wait_cycles":5632,"thread_start_us":90,"threads_started":1}
I20260927 10:48:55.889462 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:41315
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:38161
--webserver_port=38757
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:48:55.889631 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.874485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:36313 (local address 127.9.39.188:40695)
0927 10:48:55.874766 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.874770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.874786 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:48:55.875264 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.875267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.875339 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.875443 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.876589 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.877428 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.878931 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.879293 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879401 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.885098 (+ 5697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.885105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.885111 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.885143 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.887235 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.887743 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.887750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.887754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.887854 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.888162 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.888168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.888173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.888421 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.888515 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.889307 (+ 792us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.889446 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260927 10:48:55.890020 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.874120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:34757 (local address 127.9.39.190:38161)
0927 10:48:55.874249 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:55.874253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:55.874362 (+ 109us) server_negotiation.cc:407] Connection header received
0927 10:48:55.875050 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:55.875056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:55.875109 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:55.875194 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:55.876604 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.877398 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879074 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.879294 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879415 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.883516 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:55.883520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:55.883523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:55.883550 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:55.885910 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.886637 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.886641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.886644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.886751 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:55.887028 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:55.887031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:55.887032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:55.887236 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:55.887319 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:55.889731 (+ 2412us) server_negotiation.cc:299] Negotiation successful
0927 10:48:55.889872 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:48:55.889164 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.874376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.9.39.129:36313)
0927 10:48:55.874645 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.874658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.874763 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.875461 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.875464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.875879 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.876451 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.876465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.877566 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.877569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.878805 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.878815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.879534 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.879657 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.883330 (+ 3673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.883349 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.884931 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.887361 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.887367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.887372 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.887633 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.887955 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.887961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.887965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.888058 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.888563 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.888566 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.888662 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.888750 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.888924 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":198,"spinlock_wait_cycles":7040,"thread_start_us":123,"threads_started":1}
I20260927 10:48:55.890641 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:55.873106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.9.39.129:33427)
0927 10:48:55.873572 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:55.873612 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:55.874742 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:55.875337 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:55.875346 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:55.875891 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:55.876465 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.876473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.877566 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.877569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.878937 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:55.878947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:55.879584 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:55.879587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:55.879679 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:55.880525 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:55.880542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:55.883320 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:55.885702 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.885710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.885730 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.886111 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.886439 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:55.886442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:55.886444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:55.886628 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:55.887177 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:55.887183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:55.887525 (+ 342us) client_negotiation.cc:769] Sending connection context
0927 10:48:55.888515 (+ 990us) client_negotiation.cc:240] Negotiation successful
0927 10:48:55.888651 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":378,"spinlock_wait_cycles":47744,"thread_start_us":121,"threads_started":1}
I20260927 10:48:55.891924 16445 heartbeater.cc:347] Connected to a master server at 127.9.39.190:38161
I20260927 10:48:55.891997 16446 heartbeater.cc:347] Connected to a master server at 127.9.39.189:38161
I20260927 10:48:55.892061 16444 heartbeater.cc:347] Connected to a master server at 127.9.39.188:40695
I20260927 10:48:55.892165 16446 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:55.892164 16444 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:55.892165 16445 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:55.892781 16446 heartbeater.cc:511] Master 127.9.39.189:38161 requested a full tablet report, sending...
I20260927 10:48:55.892792 16444 heartbeater.cc:511] Master 127.9.39.188:40695 requested a full tablet report, sending...
I20260927 10:48:55.894394 16265 ts_manager.cc:194] Registered new tserver with Master: fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129:33189)
I20260927 10:48:55.892781 16445 heartbeater.cc:511] Master 127.9.39.190:38161 requested a full tablet report, sending...
I20260927 10:48:55.895393 16073 ts_manager.cc:194] Registered new tserver with Master: fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129:33189)
I20260927 10:48:55.895956 15473 ts_manager.cc:194] Re-registered known tserver with Master: fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129:33189)
I20260927 10:48:55.896490 15473 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:34757
W20260927 10:48:56.001627 16452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:56.001945 16452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:56.001997 16452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:56.005873 16452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:56.005923 16452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:56.005977 16452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:48:56.011006 16452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:41315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=38757
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:56.011898 16452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:56.012787 16452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:48:56.020071 15932 heartbeater.cc:503] Master 127.9.39.190:38161 was elected leader, sending a full tablet report...
W20260927 10:48:56.022202 16458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:56.022172 16457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:56.022557 16452 server_base.cc:1034] running on GCE node
W20260927 10:48:56.022181 16460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:56.023059 16452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:56.023707 16452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:56.024926 16452 hybrid_clock.cc:648] HybridClock initialized: now 1790506136024902 us; error 28 us; skew 500 ppm
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:56.028017 16452 init.cc:377] Logged in from keytab as oryx/127.9.39.130@KRBTEST.COM (short username oryx)
I20260927 10:48:56.029037 16452 webserver.cc:460] Webserver started at http://127.9.39.130:38757/ using document root <none> and password file <none>
I20260927 10:48:56.029577 16452 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:56.029651 16452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:56.033531 16452 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:48:56.035696 16467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:56.037022 16452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:48:56.037165 16452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "abe31483246640728ab67024a8b71630"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "6be15dd02166680bdc66c1445f03285a"
server_key_iv: "1d0da9c912883177ea12b663e601136b"
server_key_version: "encryptionkey@0"
I20260927 10:48:56.037762 16452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:56.046991 16452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:56.049660 16452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:56.049860 16452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:56.050417 16452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:56.051379 16452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:56.051450 16452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:56.051513 16452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:56.051565 16452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:56.060822 16452 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:41315
I20260927 10:48:56.060851 16580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:41315 every 8 connection(s)
I20260927 10:48:56.061806 16452 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:48:56.067142 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 16452
I20260927 10:48:56.067332 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15803
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.188@KRBTEST.COM
I20260927 10:48:56.074798 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:44767
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:38161
--webserver_port=45485
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:48:56.080096 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.063560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:56363 (local address 127.9.39.188:40695)
0927 10:48:56.063705 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.063709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.064595 (+ 886us) server_negotiation.cc:407] Connection header received
0927 10:48:56.065712 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.065715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.065777 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.065887 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.067450 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.068355 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.071834 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.072248 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.072377 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.075507 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.075514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.075520 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.075555 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.077623 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.078164 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.078167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.078171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.078288 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.078799 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.078802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.078804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.079036 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.079135 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.079771 (+ 636us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.079915 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 10:48:56.080165 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.063607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:46185 (local address 127.9.39.190:38161)
0927 10:48:56.063761 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.063766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.064596 (+ 830us) server_negotiation.cc:407] Connection header received
0927 10:48:56.065728 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.065731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.065783 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.065836 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.067405 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.068232 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.070303 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.070645 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.070747 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.075555 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.075559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.075562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.075591 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.077757 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.078279 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.078286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.078290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.078381 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.078776 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.078782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.078787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.079039 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.079118 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.079790 (+ 672us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.079926 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 10:48:56.080916 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.064001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.9.39.130:56363)
0927 10:48:56.064447 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.064499 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.065515 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.066055 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.066062 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.066537 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:56.067293 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.067304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.068528 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.068534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.071671 (+ 3137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.071685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.072546 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.072550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.072641 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.073193 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:56.073210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:56.075298 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:56.077734 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.077740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.077751 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.078039 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.078413 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.078419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.078423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.078630 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.079190 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:56.079197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:56.079522 (+ 325us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.079717 (+ 195us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.080051 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":2006,"spinlock_wait_cycles":12288,"thread_start_us":109,"threads_started":1}
I20260927 10:48:56.081202 16586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.064000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:46185)
0927 10:48:56.064447 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.064499 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.065535 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.066057 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.066062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.066561 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:56.067262 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.067268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.068761 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.068766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.070164 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.070175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.070949 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.070953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.071255 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.072033 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:56.072059 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:56.075385 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:56.077856 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.077860 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.077864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.078156 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.078497 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.078515 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.078520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.078630 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.079198 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:56.079203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:56.079549 (+ 346us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.080315 (+ 766us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.080444 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":330,"spinlock_wait_cycles":36224,"thread_start_us":147,"threads_started":1}
I20260927 10:48:56.082334 16581 heartbeater.cc:347] Connected to a master server at 127.9.39.188:40695
I20260927 10:48:56.082604 16581 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:56.082346 16582 heartbeater.cc:347] Connected to a master server at 127.9.39.190:38161
I20260927 10:48:56.082801 16582 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:56.083231 16581 heartbeater.cc:511] Master 127.9.39.188:40695 requested a full tablet report, sending...
I20260927 10:48:56.083231 16582 heartbeater.cc:511] Master 127.9.39.190:38161 requested a full tablet report, sending...
I20260927 10:48:56.084412 15473 ts_manager.cc:194] Re-registered known tserver with Master: abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:48:56.084954 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.064947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.9.39.130:43747)
0927 10:48:56.065210 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.065221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.065535 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.066057 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.066062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.066564 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:56.067236 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.067256 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.068421 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.068424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.070164 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.070175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.070949 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.070953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.071253 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.072128 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:56.072144 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:56.075298 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:56.081664 (+ 6366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.081667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.081669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.081933 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.083866 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.083869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.083871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.083958 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.084481 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:56.084483 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:56.084569 (+ 86us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.084660 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.084789 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":268,"spinlock_wait_cycles":43264,"thread_start_us":91,"threads_started":1}
I20260927 10:48:56.085142 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.065098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:43747 (local address 127.9.39.189:38161)
0927 10:48:56.065236 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.065241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.065274 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:48:56.065728 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.065731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.065783 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.065836 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.067409 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.068232 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.070303 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.070648 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.070758 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.075517 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.075521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.075524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.075553 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.079200 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.082057 (+ 2857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.082064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.082069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.082166 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.084082 (+ 1916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.084085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.084087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.084315 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.084403 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.084779 (+ 376us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.084929 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 10:48:56.085156 15473 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:46185
I20260927 10:48:56.085639 16265 ts_manager.cc:194] Registered new tserver with Master: abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:48:56.086733 16583 heartbeater.cc:347] Connected to a master server at 127.9.39.189:38161
I20260927 10:48:56.086818 16583 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:56.086952 16583 heartbeater.cc:511] Master 127.9.39.189:38161 requested a full tablet report, sending...
I20260927 10:48:56.087648 16073 ts_manager.cc:194] Registered new tserver with Master: abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
W20260927 10:48:56.192318 16588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:48:56.192627 16588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:48:56.192729 16588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:48:56.197155 16588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:48:56.197237 16588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:48:56.197326 16588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:48:56.202504 16588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:44767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=45485
--tserver_master_addrs=127.9.39.190:38161,127.9.39.189:38161,127.9.39.188:40695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:48:56.203591 16588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:48:56.204627 16588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:48:56.214176 16595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:56.214263 16594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:48:56.214162 16597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:48:56.214401 16588 server_base.cc:1034] running on GCE node
I20260927 10:48:56.214879 16588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:48:56.215481 16588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:48:56.216642 16588 hybrid_clock.cc:648] HybridClock initialized: now 1790506136216629 us; error 25 us; skew 500 ppm
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:48:56.219775 16588 init.cc:377] Logged in from keytab as oryx/127.9.39.131@KRBTEST.COM (short username oryx)
I20260927 10:48:56.220619 16588 webserver.cc:460] Webserver started at http://127.9.39.131:45485/ using document root <none> and password file <none>
I20260927 10:48:56.221175 16588 fs_manager.cc:359] Metadata directory not provided
I20260927 10:48:56.221279 16588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:48:56.225420 16588 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:48:56.227586 16604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:56.229061 16588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 10:48:56.229187 16588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "5efe62e66f6046968b0fc59352b2b122"
format_stamp: "Formatted at 2026-09-27 10:48:54 on dist-test-slave-p9q6"
server_key: "ba692b166219681513c7c1bf1fe8670f"
server_key_iv: "65190843e266fa75b2f7df16ea2f18e3"
server_key_version: "encryptionkey@0"
I20260927 10:48:56.229843 16588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:48:56.245869 16588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:48:56.249229 16588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:48:56.249445 16588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:48:56.250046 16588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:48:56.251001 16588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:48:56.251070 16588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:56.251142 16588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:48:56.251185 16588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:48:56.260417 16588 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:44767
I20260927 10:48:56.260437 16717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:44767 every 8 connection(s)
I20260927 10:48:56.261413 16588 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:48:56.262748 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 16588
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:48:56.275818 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.265551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.9.39.190:38161)
0927 10:48:56.265826 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.265831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.265842 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:48:56.265885 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.265889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.265944 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.266032 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.267176 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.267977 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.269164 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.269489 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.269567 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.272065 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.272069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.272071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.272101 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.273831 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.274307 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.274310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.274312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.274399 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.274761 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.274766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.274771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.274986 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.275110 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.275497 (+ 387us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.275632 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260927 10:48:56.277773 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.265191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.9.39.188:40695)
0927 10:48:56.266257 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.266269 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.266886 (+ 617us) server_negotiation.cc:407] Connection header received
0927 10:48:56.266928 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.266931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.266973 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.267048 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.268136 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.268920 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.270093 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.270437 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.270519 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.273034 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.273040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.273043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.273069 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.275399 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.276129 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.276132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.276134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.276214 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.276533 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.276536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.276538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.276723 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.276854 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.277453 (+ 599us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.277601 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":87,"threads_started":1}
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.188@KRBTEST.COM
Sep 27 10:48:56 dist-test-slave-p9q6 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506136, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.189@KRBTEST.COM
I20260927 10:48:56.281165 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.264840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:34535)
0927 10:48:56.265975 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.265988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.266080 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.267030 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.267033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.267233 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:56.267671 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.267689 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.268791 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.268794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.270938 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.270952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.271648 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.271652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.271837 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.272866 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:56.272892 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:56.274879 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:56.276962 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.276968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.276983 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.277348 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.277652 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.277655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.277657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.277845 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.279079 (+ 1234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:56.279086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:56.279431 (+ 345us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.279657 (+ 226us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.280026 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"mutex_wait_us":860,"thread_start_us":75,"threads_started":1}
I20260927 10:48:56.282043 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.265637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34535 (local address 127.9.39.190:38161)
0927 10:48:56.266658 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.266665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.266680 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:48:56.266763 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.266769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.266816 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.266903 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.267850 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.268617 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.271073 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.271406 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.271497 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.275072 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.275077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.275082 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.275114 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.276848 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.277457 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.277461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.277463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.277545 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.278664 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.278670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.278673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.278932 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.279026 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.281702 (+ 2676us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.281846 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":74,"threads_started":1}
I20260927 10:48:56.283558 16719 heartbeater.cc:347] Connected to a master server at 127.9.39.190:38161
I20260927 10:48:56.283679 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.265535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.9.39.189:38161)
0927 10:48:56.269631 (+ 4096us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.269636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.269650 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:48:56.269687 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.269690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.269730 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.271260 (+ 1530us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.271367 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.272434 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.273683 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.274075 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.274517 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.275623 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.275627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.275629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.275650 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.279483 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.281566 (+ 2083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.281569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.281571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.281651 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.281957 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.281960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.281962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.282162 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.283275 (+ 1113us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.283361 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.283506 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4029,"thread_start_us":82,"threads_started":1}
I20260927 10:48:56.283790 16719 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:56.283828 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.263559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:58377 (local address 127.9.39.188:40695)
0927 10:48:56.263676 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.263681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.265350 (+ 1669us) server_negotiation.cc:407] Connection header received
0927 10:48:56.265844 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.265849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.265903 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.265982 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.271195 (+ 5213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.271872 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.276435 (+ 4563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.276797 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.276860 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.280340 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.280343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.280346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.280369 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.281982 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.282537 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.282540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.282542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.282620 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.282926 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.282929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.282930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.283108 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.283224 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.283576 (+ 352us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.283680 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:48:56.283838 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.263718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.9.39.131:58377)
0927 10:48:56.265256 (+ 1538us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.265284 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.265727 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.266314 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.266325 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.266929 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:56.271044 (+ 4115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.271053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.272167 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.272172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.276312 (+ 4140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.276323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.278130 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.278133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.278221 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.278740 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:56.278755 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:56.280180 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:56.282105 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.282109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.282112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.282396 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.282741 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.282744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.282746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.282839 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.283280 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:56.283283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:56.283368 (+ 85us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.283511 (+ 143us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.283647 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1452,"mutex_wait_us":3655,"spinlock_wait_cycles":41344,"thread_start_us":141,"threads_started":1}
I20260927 10:48:56.284323 16719 heartbeater.cc:511] Master 127.9.39.190:38161 requested a full tablet report, sending...
I20260927 10:48:56.284360 16718 heartbeater.cc:347] Connected to a master server at 127.9.39.188:40695
I20260927 10:48:56.284439 16718 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:56.284576 16718 heartbeater.cc:511] Master 127.9.39.188:40695 requested a full tablet report, sending...
I20260927 10:48:56.285485 16265 ts_manager.cc:194] Registered new tserver with Master: 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767)
I20260927 10:48:56.285516 15472 ts_manager.cc:194] Re-registered known tserver with Master: 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767)
I20260927 10:48:56.279592 15473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 10:48:56.286223 15472 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:34535
W20260927 10:48:56.287470 15473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:48:56.288417 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.263210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:51325 (local address 127.9.39.189:38161)
0927 10:48:56.263373 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.263377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.264439 (+ 1062us) server_negotiation.cc:407] Connection header received
0927 10:48:56.265973 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.265979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.266058 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.266109 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.267863 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.268631 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.270345 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.270780 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.270884 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.285023 (+ 14139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.285026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.285028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.285047 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.286767 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.287177 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.287181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.287182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.287266 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.287566 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.287569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.287571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.287763 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.287846 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.288173 (+ 327us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.288283 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:48:56.288911 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.263716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.9.39.131:51325)
0927 10:48:56.264287 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.264342 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.265536 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.266314 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.266324 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.267539 (+ 1215us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:48:56.267680 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.267690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.268795 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.268798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.270247 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.270256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.278736 (+ 8480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.278741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.278847 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.279766 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:48:56.279780 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:48:56.282562 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:48:56.286883 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.286886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.286888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.287071 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.287374 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:48:56.287377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:48:56.287379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:48:56.287462 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:48:56.287918 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:48:56.287922 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:48:56.287997 (+ 75us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.288599 (+ 602us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.288737 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"thread_start_us":143,"threads_started":1}
I20260927 10:48:56.289451 16720 heartbeater.cc:347] Connected to a master server at 127.9.39.189:38161
I20260927 10:48:56.289531 16720 heartbeater.cc:464] Registering TS with master...
I20260927 10:48:56.289718 16720 heartbeater.cc:511] Master 127.9.39.189:38161 requested a full tablet report, sending...
I20260927 10:48:56.290819 16073 ts_manager.cc:194] Registered new tserver with Master: 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767)
I20260927 10:48:56.291486 16735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.286446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.0.0.1:39480)
0927 10:48:56.287418 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.287442 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.287569 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.287934 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.287938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.287960 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:56.288068 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.288075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.289140 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.289143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.290183 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.290194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.291187 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.291190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.291258 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.291277 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.291340 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.291388 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":77,"threads_started":1}
I20260927 10:48:56.291548 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.286740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.9.39.188:40695)
0927 10:48:56.287287 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.287291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.287505 (+ 214us) server_negotiation.cc:407] Connection header received
0927 10:48:56.287679 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.287683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.287740 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.287800 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:56.288185 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.289001 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.290287 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.290912 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.291031 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.291165 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.291377 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.291445 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479}
I20260927 10:48:56.292654 16275 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260927 10:48:56.293004 16738 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83399f1eda404af39814d232d3517772. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:56.293282 16738 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: LEARNER
I20260927 10:48:56.293586 16739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: NON_VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: true } } }
I20260927 10:48:56.293708 16739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:56.297263 16739 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: attempting to promote NON_VOTER d4a39675396e464a9b7fd6239123abcd to VOTER
I20260927 10:48:56.297581 16739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:56.298636 16275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 LEARNER]: Refusing update from remote peer 83399f1eda404af39814d232d3517772: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260927 10:48:56.298934 16083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Refusing update from remote peer 83399f1eda404af39814d232d3517772: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260927 10:48:56.299018 16592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260927 10:48:56.299304 16592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260927 10:48:56.300901 16592 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d4a39675396e464a9b7fd6239123abcd (127.9.39.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.301438 16275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d4a39675396e464a9b7fd6239123abcd (127.9.39.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.301789 16083 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d4a39675396e464a9b7fd6239123abcd (127.9.39.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.302476 16592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.302585 16592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:56.302727 16592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.302798 16592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:56.302884 16592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.302975 16592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:56.303103 16738 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.303203 16738 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:48:56.303434 16747 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83399f1eda404af39814d232d3517772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:56.303579 16747 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:48:56.312099 16735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.304675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:44767 (local address 127.0.0.1:47842)
0927 10:48:56.306185 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.306200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.306271 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.307924 (+ 1653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.307927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.307943 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:56.308010 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.308014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.309396 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.309400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.310575 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.310585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.311700 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.311704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.311806 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.311830 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.311873 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.311953 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1436,"mutex_wait_us":4}
I20260927 10:48:56.312108 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.304875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:33189 (local address 127.0.0.1:35578)
0927 10:48:56.306851 (+ 1976us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.306862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.306966 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.307507 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.307510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.307529 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:56.307631 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.307637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.308950 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.308953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.310194 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.310204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.311661 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.311664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.311760 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.311787 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.311835 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.311932 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1926,"thread_start_us":94,"threads_started":1}
I20260927 10:48:56.312726 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.305754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35578 (local address 127.9.39.129:33189)
0927 10:48:56.306688 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.306736 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.306954 (+ 218us) server_negotiation.cc:407] Connection header received
0927 10:48:56.307083 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.307090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.307357 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.307528 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:56.307722 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.308826 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.310667 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.311532 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.312292 (+ 760us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.312457 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.312505 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.312589 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":101,"threads_started":1}
I20260927 10:48:56.312731 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.304871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.9.39.131:44767)
0927 10:48:56.307498 (+ 2627us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.307509 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.307530 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:48:56.307596 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.307601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.307793 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.307944 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:56.308128 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.309274 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.310766 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.311568 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.312277 (+ 709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.312451 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.312500 (+ 49us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.312595 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2497,"thread_start_us":114,"threads_started":1}
I20260927 10:48:56.315245 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.305189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41315 (local address 127.0.0.1:45448)
0927 10:48:56.307140 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.307152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.307220 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.308645 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.308647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.308662 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:56.308735 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.308740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.310259 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.310264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.311804 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.311814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.314900 (+ 3086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.314904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.314990 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.315016 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.315067 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.315130 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1899,"thread_start_us":101,"threads_started":1}
I20260927 10:48:56.315616 16652 tablet_service.cc:1467] Processing CreateTablet for tablet 776147e1fb69423894ed52b58e673976 (DEFAULT_TABLE table=test-table [id=f898388c9b2945bebb13ddccd1631ac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:56.315615 16378 tablet_service.cc:1467] Processing CreateTablet for tablet 776147e1fb69423894ed52b58e673976 (DEFAULT_TABLE table=test-table [id=f898388c9b2945bebb13ddccd1631ac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:56.316072 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.305725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.9.39.130:41315)
0927 10:48:56.308222 (+ 2497us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.308232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.308252 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:48:56.308319 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.308323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.308491 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.308861 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:56.309023 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.310129 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.311941 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.314042 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.315661 (+ 1619us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.315792 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.315839 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.315923 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2396,"thread_start_us":93,"threads_started":1}
I20260927 10:48:56.316690 16378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776147e1fb69423894ed52b58e673976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:56.316701 16652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776147e1fb69423894ed52b58e673976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:56.318894 16515 tablet_service.cc:1467] Processing CreateTablet for tablet 776147e1fb69423894ed52b58e673976 (DEFAULT_TABLE table=test-table [id=f898388c9b2945bebb13ddccd1631ac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:48:56.319839 16515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776147e1fb69423894ed52b58e673976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:48:56.324203 16754 tablet_bootstrap.cc:492] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Bootstrap starting.
I20260927 10:48:56.326157 16754 tablet_bootstrap.cc:654] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:56.326946 16754 log.cc:824] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:56.327105 16753 tablet_bootstrap.cc:492] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Bootstrap starting.
I20260927 10:48:56.327795 16756 tablet_bootstrap.cc:492] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Bootstrap starting.
I20260927 10:48:56.329320 16753 tablet_bootstrap.cc:654] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:56.329933 16756 tablet_bootstrap.cc:654] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Neither blocks nor log segments found. Creating new log.
I20260927 10:48:56.330161 16753 log.cc:824] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:56.330302 16754 tablet_bootstrap.cc:492] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: No bootstrap required, opened a new log
I20260927 10:48:56.330516 16754 ts_tablet_manager.cc:1397] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260927 10:48:56.330755 16756 log.cc:824] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Log is configured to *not* fsync() on all Append() calls
I20260927 10:48:56.332264 16753 tablet_bootstrap.cc:492] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: No bootstrap required, opened a new log
I20260927 10:48:56.332331 16756 tablet_bootstrap.cc:492] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: No bootstrap required, opened a new log
I20260927 10:48:56.332506 16753 ts_tablet_manager.cc:1397] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:56.332563 16756 ts_tablet_manager.cc:1397] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:48:56.334456 16754 raft_consensus.cc:348] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.334761 16754 raft_consensus.cc:374] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:56.334815 16754 raft_consensus.cc:729] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd60eeebf2084b8baf3617e842cdd7ac, State: Initialized, Role: FOLLOWER
I20260927 10:48:56.335269 16754 consensus_queue.cc:260] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.336591 16754 ts_tablet_manager.cc:1428] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260927 10:48:56.336993 16445 heartbeater.cc:503] Master 127.9.39.190:38161 was elected leader, sending a full tablet report...
I20260927 10:48:56.336884 16756 raft_consensus.cc:348] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.336935 16753 raft_consensus.cc:348] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.337167 16756 raft_consensus.cc:374] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:56.337179 16753 raft_consensus.cc:374] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:48:56.337225 16753 raft_consensus.cc:729] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5efe62e66f6046968b0fc59352b2b122, State: Initialized, Role: FOLLOWER
I20260927 10:48:56.337275 16756 raft_consensus.cc:729] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe31483246640728ab67024a8b71630, State: Initialized, Role: FOLLOWER
I20260927 10:48:56.337649 16753 consensus_queue.cc:260] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.337674 16756 consensus_queue.cc:260] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.338527 16753 ts_tablet_manager.cc:1428] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:48:56.338837 16719 heartbeater.cc:503] Master 127.9.39.190:38161 was elected leader, sending a full tablet report...
I20260927 10:48:56.339512 16756 ts_tablet_manager.cc:1428] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 10:48:56.342370 16582 heartbeater.cc:503] Master 127.9.39.190:38161 was elected leader, sending a full tablet report...
W20260927 10:48:56.372655 16447 tablet.cc:2369] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:48:56.415771 16104 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.9.39.189:38161
I20260927 10:48:56.473279 16759 raft_consensus.cc:482] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:56.473511 16759 raft_consensus.cc:504] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.474817 16759 leader_election.cc:288] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767), abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:48:56.478880 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.475097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:44767 (local address 127.9.39.129:42971)
0927 10:48:56.475330 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.475345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.475484 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.475797 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.475802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.475859 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:56.475968 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.475976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.476754 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.476757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.477627 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.477636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.478310 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.478314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.478445 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.478472 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.478572 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.478675 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260927 10:48:56.478909 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.475584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:36431 (local address 127.9.39.130:41315)
0927 10:48:56.475748 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.475754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.475772 (+ 18us) server_negotiation.cc:407] Connection header received
0927 10:48:56.475904 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.475907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.475955 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.476045 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:56.476272 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.476802 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.477829 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.478200 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.478335 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.478387 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.478733 (+ 346us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.478802 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:48:56.478941 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.475235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:42971 (local address 127.9.39.131:44767)
0927 10:48:56.475403 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.475408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.475431 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:48:56.475596 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.475600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.475658 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.475758 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:56.476072 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.476645 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.477750 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.478188 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.478313 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.478479 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.478607 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.478693 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 10:48:56.478961 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.475439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41315 (local address 127.9.39.129:36431)
0927 10:48:56.475721 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:56.475736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:56.475802 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:56.476067 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:56.476070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:56.476086 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:56.476165 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.476170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.476931 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.476933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.477705 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:56.477716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.478331 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.478334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:56.478529 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.478560 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 10:48:56.478721 (+ 161us) client_negotiation.cc:240] Negotiation successful
0927 10:48:56.478801 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":51,"thread_start_us":75,"threads_started":1}
I20260927 10:48:56.479568 16535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776147e1fb69423894ed52b58e673976" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe31483246640728ab67024a8b71630" is_pre_election: true
I20260927 10:48:56.479573 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776147e1fb69423894ed52b58e673976" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5efe62e66f6046968b0fc59352b2b122" is_pre_election: true
I20260927 10:48:56.479849 16535 raft_consensus.cc:2454] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 0.
I20260927 10:48:56.479858 16672 raft_consensus.cc:2454] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 0.
I20260927 10:48:56.480265 16334 leader_election.cc:302] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe31483246640728ab67024a8b71630, fd60eeebf2084b8baf3617e842cdd7ac; no voters:
I20260927 10:48:56.480527 16759 raft_consensus.cc:2781] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:48:56.480618 16759 raft_consensus.cc:482] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:56.480672 16759 raft_consensus.cc:3037] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:56.481626 16759 raft_consensus.cc:504] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.481980 16759 leader_election.cc:288] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 election: Requested vote from peers 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767), abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:48:56.482343 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776147e1fb69423894ed52b58e673976" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5efe62e66f6046968b0fc59352b2b122"
I20260927 10:48:56.482499 16672 raft_consensus.cc:3037] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:56.482514 16535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776147e1fb69423894ed52b58e673976" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe31483246640728ab67024a8b71630"
I20260927 10:48:56.482649 16535 raft_consensus.cc:3037] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:48:56.483747 16672 raft_consensus.cc:2454] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 1.
I20260927 10:48:56.483717 16535 raft_consensus.cc:2454] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 1.
I20260927 10:48:56.484033 16334 leader_election.cc:302] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe31483246640728ab67024a8b71630, fd60eeebf2084b8baf3617e842cdd7ac; no voters:
I20260927 10:48:56.484184 16759 raft_consensus.cc:2781] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:48:56.484419 16759 raft_consensus.cc:686] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 1 LEADER]: Becoming Leader. State: Replica: fd60eeebf2084b8baf3617e842cdd7ac, State: Running, Role: LEADER
I20260927 10:48:56.484768 16759 consensus_queue.cc:237] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:48:56.488091 15471 catalog_manager.cc:5495] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac reported cstate change: term changed from 0 to 1, leader changed from <none> to fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129). New cstate: current_term: 1 leader_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } health_report { overall_health: UNKNOWN } } }
I20260927 10:48:56.501645 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.498179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.9.39.129:33189)
0927 10:48:56.498313 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.498316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.498343 (+ 27us) server_negotiation.cc:407] Connection header received
0927 10:48:56.498405 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.498407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.498456 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.498540 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:48:56.498813 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.499323 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.500172 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.500395 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.500523 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.500856 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:48:56.501280 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:48:56.501408 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.501453 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.501505 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:48:56.507160 16535 raft_consensus.cc:1261] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Refusing update from remote peer fd60eeebf2084b8baf3617e842cdd7ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:56.507185 16672 raft_consensus.cc:1261] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Refusing update from remote peer fd60eeebf2084b8baf3617e842cdd7ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:48:56.507885 16759 consensus_queue.cc:1035] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:48:56.508038 16764 consensus_queue.cc:1035] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 10:48:56.512756 16721 tablet.cc:2369] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:48:56.521436 16768 mvcc.cc:205] Tried to move back new op lower bound from 7333913135126437888 to 7333913135044173824. Current Snapshot: MvccSnapshot[applied={T|T < 7333913135126437888 or (T in {7333913135126437888})}]
I20260927 10:48:56.522245 16769 mvcc.cc:205] Tried to move back new op lower bound from 7333913135126437888 to 7333913135044173824. Current Snapshot: MvccSnapshot[applied={T|T < 7333913135126437888}]
I20260927 10:48:56.525815 15471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:56.525969 15471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:48:56.530644 15471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83399f1eda404af39814d232d3517772: Sending DeleteTablet for 3 replicas of tablet 776147e1fb69423894ed52b58e673976
I20260927 10:48:56.531404 16652 tablet_service.cc:1514] Processing DeleteTablet for tablet 776147e1fb69423894ed52b58e673976 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:56 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:47842
I20260927 10:48:56.531467 16378 tablet_service.cc:1514] Processing DeleteTablet for tablet 776147e1fb69423894ed52b58e673976 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:56 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:35578
I20260927 10:48:56.531549 16515 tablet_service.cc:1514] Processing DeleteTablet for tablet 776147e1fb69423894ed52b58e673976 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:48:56 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:45448
I20260927 10:48:56.531939 16777 tablet_replica.cc:330] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: stopping tablet replica
I20260927 10:48:56.531953 16779 tablet_replica.cc:330] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: stopping tablet replica
I20260927 10:48:56.532248 16779 raft_consensus.cc:2229] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:56.532269 16777 raft_consensus.cc:2229] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:48:56.532562 16779 raft_consensus.cc:2258] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:56.532765 16777 raft_consensus.cc:2258] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 10:48:56.533478 16108 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.9.39.189:38161: Invalid argument: socket address is ambiguous
I20260927 10:48:56.534314 16779 ts_tablet_manager.cc:1905] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:56.534402 16777 ts_tablet_manager.cc:1905] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:56.534974 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15441
I20260927 10:48:56.533100 16778 tablet_replica.cc:330] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: stopping tablet replica
I20260927 10:48:56.536943 16778 raft_consensus.cc:2229] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:48:56.537326 16778 raft_consensus.cc:2258] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:48:56.538470 16777 ts_tablet_manager.cc:1918] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:56.538616 16777 log.cc:1196] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/776147e1fb69423894ed52b58e673976
I20260927 10:48:56.538735 16779 ts_tablet_manager.cc:1918] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:56.538829 16779 log.cc:1196] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/776147e1fb69423894ed52b58e673976
I20260927 10:48:56.539150 16779 ts_tablet_manager.cc:1939] T 776147e1fb69423894ed52b58e673976 P abe31483246640728ab67024a8b71630: Deleting consensus metadata
I20260927 10:48:56.539152 16777 ts_tablet_manager.cc:1939] T 776147e1fb69423894ed52b58e673976 P 5efe62e66f6046968b0fc59352b2b122: Deleting consensus metadata
W20260927 10:48:56.542562 16471 connection.cc:535] server connection from 127.0.0.1:45448 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 10:48:56.542660 16607 connection.cc:535] server connection from 127.0.0.1:47842 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 10:48:56.548187 16778 ts_tablet_manager.cc:1905] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:48:56.551311 16778 ts_tablet_manager.cc:1918] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:48:56.551427 16778 log.cc:1196] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/776147e1fb69423894ed52b58e673976
I20260927 10:48:56.551805 16778 ts_tablet_manager.cc:1939] T 776147e1fb69423894ed52b58e673976 P fd60eeebf2084b8baf3617e842cdd7ac: Deleting consensus metadata
W20260927 10:48:56.552983 16333 connection.cc:441] server connection from 127.0.0.1:35578 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35578 (request call id 1) could send its response
I20260927 10:48:56.555083 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.533696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.9.39.189:38161)
0927 10:48:56.533805 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.533810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.543947 (+ 10137us) server_negotiation.cc:407] Connection header received
0927 10:48:56.544101 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.544106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.544161 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.544262 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.545088 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.545615 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.546754 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.547101 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.547213 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.548975 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.548982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.548985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.549010 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.550793 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.551636 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.551641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.551643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.551732 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.552066 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.552290 (+ 224us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.552292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.552506 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.552635 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.554362 (+ 1727us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.554884 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 10:48:56.557000 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.546537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.9.39.188:40695)
0927 10:48:56.546648 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.546653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.547116 (+ 463us) server_negotiation.cc:407] Connection header received
0927 10:48:56.547402 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.547405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.547457 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.547510 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.549185 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.549962 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.551177 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.551536 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.551618 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.553319 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.553323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.553325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.553357 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.555137 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.555596 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.555602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.555606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.555690 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.556002 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.556009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.556013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.556212 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.556317 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.556690 (+ 373us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.556830 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:48:56.558192 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.546518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.9.39.189:38161)
0927 10:48:56.546643 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.546648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.547233 (+ 585us) server_negotiation.cc:407] Connection header received
0927 10:48:56.547396 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.547399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.547441 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.547494 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.548559 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.549349 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.550445 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.550841 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.550922 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.552659 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.552665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.552669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.552700 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.554786 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.556662 (+ 1876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.556665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.556667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.556744 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.557036 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.557039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.557041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.557247 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.557363 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.557919 (+ 556us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.558035 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 10:48:56.561529 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:56.533910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39484 (local address 127.9.39.188:40695)
0927 10:48:56.534012 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:56.534015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:56.553401 (+ 19386us) server_negotiation.cc:407] Connection header received
0927 10:48:56.553443 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:56.553446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:56.553490 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:56.553554 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:48:56.554534 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.555314 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.556426 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:56.556757 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:56.556829 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:56.558482 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:48:56.558486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:48:56.558488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:48:56.558512 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:48:56.560070 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.560454 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.560458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.560460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.560540 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:48:56.560791 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:48:56.560794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:48:56.560796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:48:56.561013 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:48:56.561136 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:56.561281 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 10:48:56.561383 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:48:57.519524 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:57.518929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:56219)
0927 10:48:57.519254 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:57.519396 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":78,"threads_started":1}
W20260927 10:48:57.519913 16719 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:38161 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:57.521654 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:57.521158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:38357)
0927 10:48:57.521443 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:57.521545 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
W20260927 10:48:57.522276 16445 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:38161 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:57.525777 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:57.525116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:58157)
0927 10:48:57.525436 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:57.525545 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":74,"threads_started":1}
W20260927 10:48:57.526182 16582 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:38161 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:58.030095 16791 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83399f1eda404af39814d232d3517772)
I20260927 10:48:58.030282 16792 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83399f1eda404af39814d232d3517772)
I20260927 10:48:58.030354 16791 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:58.030471 16792 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:58.031478 16792 leader_election.cc:288] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83399f1eda404af39814d232d3517772 (127.9.39.190:38161), b5bad59ad42147f898036f6467d37c28 (127.9.39.189:38161)
I20260927 10:48:58.031616 16791 leader_election.cc:288] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83399f1eda404af39814d232d3517772 (127.9.39.190:38161), d4a39675396e464a9b7fd6239123abcd (127.9.39.188:40695)
I20260927 10:48:58.032195 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.031587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50714)
0927 10:48:58.032002 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.032086 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":61,"threads_started":1}
I20260927 10:48:58.032295 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.031591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50726)
0927 10:48:58.032139 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.032194 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":59,"threads_started":1}
W20260927 10:48:58.032433 16060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
W20260927 10:48:58.032440 16252 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:58.033305 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.033084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50728)
0927 10:48:58.033201 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.033244 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260927 10:48:58.033418 16252 leader_election.cc:334] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:58.033619 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.033189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50734)
0927 10:48:58.033311 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.033461 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260927 10:48:58.033711 16060 leader_election.cc:334] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:58.036098 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.031594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:40695 (local address 127.0.0.1:39498)
0927 10:48:58.032002 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.032017 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:58.032114 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:58.032627 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:58.032630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:58.032654 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:58.032786 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:58.032793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.034008 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.034011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:58.034945 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:58.034953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.035599 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.035603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:58.035722 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:58.035764 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 10:48:58.035859 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 10:48:58.035930 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":154,"spinlock_wait_cycles":3072,"thread_start_us":60,"threads_started":1}
I20260927 10:48:58.036096 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.031815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.9.39.188:40695)
0927 10:48:58.032358 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:58.032363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:58.032379 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:48:58.032424 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:58.032427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:58.032488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:58.032899 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:58.033036 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.033889 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.035056 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.035466 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.035600 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:58.035657 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:58.035875 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 10:48:58.035945 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"spinlock_wait_cycles":5376,"thread_start_us":69,"threads_started":1}
I20260927 10:48:58.036139 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.031898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38161 (local address 127.0.0.1:40966)
0927 10:48:58.032315 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.032330 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:48:58.032523 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:48:58.032917 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:48:58.032919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:48:58.032939 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:48:58.033081 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:58.033092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.034079 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.034082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:58.035026 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:48:58.035035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.035679 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.035684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:48:58.035777 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:58.035814 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 10:48:58.035909 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 10:48:58.035984 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"spinlock_wait_cycles":115328,"thread_start_us":49,"threads_started":1}
I20260927 10:48:58.036139 16798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.031995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.9.39.189:38161)
0927 10:48:58.032212 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 10:48:58.032217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:48:58.032391 (+ 174us) server_negotiation.cc:407] Connection header received
0927 10:48:58.032652 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:48:58.032656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:48:58.032707 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:48:58.032778 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:48:58.033219 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.033965 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.035147 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:48:58.035544 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:48:58.035712 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:48:58.035789 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 10:48:58.035923 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 10:48:58.035992 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":160,"spinlock_wait_cycles":384,"thread_start_us":57,"threads_started":1}
I20260927 10:48:58.036760 16275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5bad59ad42147f898036f6467d37c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d4a39675396e464a9b7fd6239123abcd" is_pre_election: true
I20260927 10:48:58.036780 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4a39675396e464a9b7fd6239123abcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b5bad59ad42147f898036f6467d37c28" is_pre_election: true
I20260927 10:48:58.036993 16275 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bad59ad42147f898036f6467d37c28 in term 1.
I20260927 10:48:58.036999 16083 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4a39675396e464a9b7fd6239123abcd in term 1.
I20260927 10:48:58.037341 16057 leader_election.cc:302] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5bad59ad42147f898036f6467d37c28, d4a39675396e464a9b7fd6239123abcd; no voters: 83399f1eda404af39814d232d3517772
I20260927 10:48:58.037380 16252 leader_election.cc:302] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5bad59ad42147f898036f6467d37c28, d4a39675396e464a9b7fd6239123abcd; no voters: 83399f1eda404af39814d232d3517772
I20260927 10:48:58.037572 16791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 10:48:58.037631 16791 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83399f1eda404af39814d232d3517772)
I20260927 10:48:58.037671 16791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 1 FOLLOWER]: Advancing to term 2
I20260927 10:48:58.038792 16792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 10:48:58.038857 16792 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83399f1eda404af39814d232d3517772)
I20260927 10:48:58.038798 16791 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:58.038897 16792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 1 FOLLOWER]: Advancing to term 2
I20260927 10:48:58.039202 16791 leader_election.cc:288] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 2 election: Requested vote from peers 83399f1eda404af39814d232d3517772 (127.9.39.190:38161), d4a39675396e464a9b7fd6239123abcd (127.9.39.188:40695)
I20260927 10:48:58.039502 16275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5bad59ad42147f898036f6467d37c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d4a39675396e464a9b7fd6239123abcd"
I20260927 10:48:58.039592 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.039345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50748)
0927 10:48:58.039479 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.039533 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260927 10:48:58.040292 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.039936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50758)
0927 10:48:58.040039 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.040244 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 10:48:58.040287 16792 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
W20260927 10:48:58.040422 16060 leader_election.cc:334] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:58.040565 16275 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bad59ad42147f898036f6467d37c28 in current term 2: Already voted for candidate d4a39675396e464a9b7fd6239123abcd in this term.
I20260927 10:48:58.040709 16792 leader_election.cc:288] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [CANDIDATE]: Term 2 election: Requested vote from peers 83399f1eda404af39814d232d3517772 (127.9.39.190:38161), b5bad59ad42147f898036f6467d37c28 (127.9.39.189:38161)
I20260927 10:48:58.041080 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.040811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50774)
0927 10:48:58.040933 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.040989 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 10:48:58.041186 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4a39675396e464a9b7fd6239123abcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b5bad59ad42147f898036f6467d37c28"
I20260927 10:48:58.041214 16057 leader_election.cc:302] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bad59ad42147f898036f6467d37c28; no voters: 83399f1eda404af39814d232d3517772, d4a39675396e464a9b7fd6239123abcd
I20260927 10:48:58.041369 16083 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d4a39675396e464a9b7fd6239123abcd in current term 2: Already voted for candidate b5bad59ad42147f898036f6467d37c28 in this term.
I20260927 10:48:58.041450 16791 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 10:48:58.041851 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.041627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:50776)
0927 10:48:58.041732 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.041779 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 10:48:58.041960 16252 leader_election.cc:334] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:58.042018 16252 leader_election.cc:302] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4a39675396e464a9b7fd6239123abcd; no voters: 83399f1eda404af39814d232d3517772, b5bad59ad42147f898036f6467d37c28
I20260927 10:48:58.042192 16792 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 10:48:58.521270 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.520784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:45787)
0927 10:48:58.521087 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.521156 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
I20260927 10:48:58.523489 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.522839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:53089)
0927 10:48:58.523327 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.523384 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":88,"threads_started":1}
I20260927 10:48:58.527160 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:58.526788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:54833)
0927 10:48:58.527008 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:58.527054 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260927 10:48:59.523063 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.522573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:54313)
0927 10:48:59.522870 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.522948 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":148,"threads_started":1}
I20260927 10:48:59.524559 16805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.524124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:58953)
0927 10:48:59.524357 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.524433 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260927 10:48:59.528232 16806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.527893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:50755)
0927 10:48:59.528077 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.528122 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260927 10:48:59.933997 16807 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:48:59.934116 16807 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:59.934552 16807 leader_election.cc:288] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83399f1eda404af39814d232d3517772 (127.9.39.190:38161), d4a39675396e464a9b7fd6239123abcd (127.9.39.188:40695)
I20260927 10:48:59.934989 16275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5bad59ad42147f898036f6467d37c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d4a39675396e464a9b7fd6239123abcd" is_pre_election: true
I20260927 10:48:59.935124 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.934752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:51050)
0927 10:48:59.934991 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.935045 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260927 10:48:59.935144 16275 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bad59ad42147f898036f6467d37c28 in term 2.
I20260927 10:48:59.935485 16057 leader_election.cc:302] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5bad59ad42147f898036f6467d37c28, d4a39675396e464a9b7fd6239123abcd; no voters:
I20260927 10:48:59.935708 16807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 10:48:59.935772 16807 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:48:59.935815 16807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 2 FOLLOWER]: Advancing to term 3
I20260927 10:48:59.935946 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.935684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:51064)
0927 10:48:59.935818 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.935879 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 10:48:59.936105 16060 leader_election.cc:334] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:59.936975 16807 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:59.937340 16807 leader_election.cc:288] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 3 election: Requested vote from peers 83399f1eda404af39814d232d3517772 (127.9.39.190:38161), d4a39675396e464a9b7fd6239123abcd (127.9.39.188:40695)
I20260927 10:48:59.937703 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.937407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:51072)
0927 10:48:59.937555 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.937643 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260927 10:48:59.937677 16275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5bad59ad42147f898036f6467d37c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d4a39675396e464a9b7fd6239123abcd"
I20260927 10:48:59.937809 16275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 2 FOLLOWER]: Advancing to term 3
I20260927 10:48:59.938369 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:48:59.938190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:51084)
0927 10:48:59.938287 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 10:48:59.938328 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 10:48:59.938525 16060 leader_election.cc:334] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
I20260927 10:48:59.938998 16275 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bad59ad42147f898036f6467d37c28 in term 3.
I20260927 10:48:59.939255 16057 leader_election.cc:302] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5bad59ad42147f898036f6467d37c28, d4a39675396e464a9b7fd6239123abcd; no voters: 83399f1eda404af39814d232d3517772
I20260927 10:48:59.939427 16807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 10:48:59.939599 16807 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [term 3 LEADER]: Becoming Leader. State: Replica: b5bad59ad42147f898036f6467d37c28, State: Running, Role: LEADER
I20260927 10:48:59.939851 16807 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } }
I20260927 10:48:59.940843 16810 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5bad59ad42147f898036f6467d37c28. Latest consensus state: current_term: 3 leader_uuid: "b5bad59ad42147f898036f6467d37c28" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:48:59.940958 16810 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: LEADER
I20260927 10:48:59.941228 16812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:48:59.943478 16812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:48:59.943892 16812 catalog_manager.cc:1259] Loaded cluster ID: 9eecf9d5c0404e83a1640903632c6a4e
I20260927 10:48:59.943959 16812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:48:59.944506 16812 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:48:59.944887 16812 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Loaded TSK: 0
I20260927 10:48:59.945542 16812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:49:00.020073 16073 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:49:00.023217 16073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:00.026427 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.026175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:51106)
0927 10:49:00.026305 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.026362 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 10:49:00.026620 16275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [term 3 FOLLOWER]: Refusing update from remote peer b5bad59ad42147f898036f6467d37c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260927 10:49:00.027083 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.026880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.0.0.1:51120)
0927 10:49:00.026980 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.027032 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 10:49:00.027179 16810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260927 10:49:00.027426 16060 consensus_peers.cc:487] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 -> Peer 83399f1eda404af39814d232d3517772 (127.9.39.190:38161): Couldn't send request to peer 83399f1eda404af39814d232d3517772. Status: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 10:49:00.028769 16814 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5bad59ad42147f898036f6467d37c28. Latest consensus state: current_term: 3 leader_uuid: "b5bad59ad42147f898036f6467d37c28" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:49:00.028896 16814 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:49:00.030251 16810 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b5bad59ad42147f898036f6467d37c28" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:49:00.030370 16810 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:00.030579 16814 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b5bad59ad42147f898036f6467d37c28" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "83399f1eda404af39814d232d3517772" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 38161 } } peers { permanent_uuid: "b5bad59ad42147f898036f6467d37c28" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38161 } attrs { promote: false } } peers { permanent_uuid: "d4a39675396e464a9b7fd6239123abcd" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 40695 } attrs { promote: false } } }
I20260927 10:49:00.030751 16814 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a39675396e464a9b7fd6239123abcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:49:00.041606 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.037287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:33189 (local address 127.0.0.1:35590)
0927 10:49:00.038068 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.038081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:00.038169 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:00.038412 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:00.038415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:00.038442 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:00.038547 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:00.038552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.039505 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.039509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:00.040360 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:00.040365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.041155 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.041159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:00.041265 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.041293 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 10:49:00.041380 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 10:49:00.041462 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":28,"spinlock_wait_cycles":2944,"thread_start_us":82,"threads_started":1}
I20260927 10:49:00.041606 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.037493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.9.39.129:33189)
0927 10:49:00.037760 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:00.037767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:00.038158 (+ 391us) server_negotiation.cc:407] Connection header received
0927 10:49:00.038257 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:00.038259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:00.038298 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:00.038399 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:00.038652 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.039392 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.040478 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.041037 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.041157 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.041224 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:00.041408 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 10:49:00.041493 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260927 10:49:00.041827 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.037092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:44767 (local address 127.0.0.1:47856)
0927 10:49:00.037189 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.037206 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:00.037300 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:00.037792 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:00.037795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:00.037813 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:00.037935 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:00.037942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.039064 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.039069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:00.040341 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:00.040351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.041483 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.041487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:00.041568 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.041590 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:49:00.041635 (+ 45us) client_negotiation.cc:240] Negotiation successful
0927 10:49:00.041709 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":58}
I20260927 10:49:00.042395 16378 tablet_service.cc:1467] Processing CreateTablet for tablet 31736a4cc8a140aa9813956a5a51a8bd (DEFAULT_TABLE table=test-table [id=48c72823a0ae42fe9cbc6b3a1544b27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:00.042604 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.037229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47856 (local address 127.9.39.131:44767)
0927 10:49:00.037492 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:00.037496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:00.037510 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:49:00.037565 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:00.037571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:00.037625 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:00.038053 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:00.038177 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.038942 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.040538 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.041347 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.042273 (+ 926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.042334 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:00.042370 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 10:49:00.042443 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
I20260927 10:49:00.042721 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.037596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41315 (local address 127.0.0.1:45456)
0927 10:49:00.038353 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.038364 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:00.038450 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:00.038957 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:00.038959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:00.038981 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:00.039091 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:00.039096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.040114 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.040118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:00.041317 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:00.041330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.042290 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.042293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:00.042373 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.042395 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:49:00.042478 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 10:49:00.042544 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"spinlock_wait_cycles":4864,"thread_start_us":74,"threads_started":1}
I20260927 10:49:00.042727 16378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31736a4cc8a140aa9813956a5a51a8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:00.042960 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.037809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.9.39.130:41315)
0927 10:49:00.038037 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:00.038040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:00.038664 (+ 624us) server_negotiation.cc:407] Connection header received
0927 10:49:00.038795 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:00.038799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:00.038848 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:00.038933 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:00.039222 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.039996 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.041466 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.042076 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.042284 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.042336 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:00.042505 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 10:49:00.042571 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
I20260927 10:49:00.043144 16652 tablet_service.cc:1467] Processing CreateTablet for tablet 31736a4cc8a140aa9813956a5a51a8bd (DEFAULT_TABLE table=test-table [id=48c72823a0ae42fe9cbc6b3a1544b27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:00.043376 16652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31736a4cc8a140aa9813956a5a51a8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:00.043429 16515 tablet_service.cc:1467] Processing CreateTablet for tablet 31736a4cc8a140aa9813956a5a51a8bd (DEFAULT_TABLE table=test-table [id=48c72823a0ae42fe9cbc6b3a1544b27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:00.043674 16515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31736a4cc8a140aa9813956a5a51a8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:00.045418 16825 tablet_bootstrap.cc:492] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac: Bootstrap starting.
I20260927 10:49:00.046085 16827 tablet_bootstrap.cc:492] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Bootstrap starting.
I20260927 10:49:00.046882 16827 tablet_bootstrap.cc:654] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:00.046882 16825 tablet_bootstrap.cc:654] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:00.046885 16826 tablet_bootstrap.cc:492] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Bootstrap starting.
I20260927 10:49:00.047659 16826 tablet_bootstrap.cc:654] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:00.048259 16827 tablet_bootstrap.cc:492] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: No bootstrap required, opened a new log
I20260927 10:49:00.048350 16827 ts_tablet_manager.cc:1397] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 10:49:00.048444 16825 tablet_bootstrap.cc:492] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac: No bootstrap required, opened a new log
I20260927 10:49:00.048529 16825 ts_tablet_manager.cc:1397] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:49:00.048790 16827 raft_consensus.cc:348] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.048875 16826 tablet_bootstrap.cc:492] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: No bootstrap required, opened a new log
I20260927 10:49:00.048911 16827 raft_consensus.cc:374] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:00.048946 16827 raft_consensus.cc:729] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe31483246640728ab67024a8b71630, State: Initialized, Role: FOLLOWER
I20260927 10:49:00.048905 16825 raft_consensus.cc:348] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.048971 16826 ts_tablet_manager.cc:1397] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 10:49:00.049005 16825 raft_consensus.cc:374] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:00.049047 16825 raft_consensus.cc:729] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd60eeebf2084b8baf3617e842cdd7ac, State: Initialized, Role: FOLLOWER
I20260927 10:49:00.049103 16827 consensus_queue.cc:260] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.049157 16825 consensus_queue.cc:260] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.049436 16827 ts_tablet_manager.cc:1428] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:00.049393 16826 raft_consensus.cc:348] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.049502 16826 raft_consensus.cc:374] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:00.049544 16826 raft_consensus.cc:729] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5efe62e66f6046968b0fc59352b2b122, State: Initialized, Role: FOLLOWER
I20260927 10:49:00.049678 16826 consensus_queue.cc:260] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.049785 16825 ts_tablet_manager.cc:1428] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:00.050031 16826 ts_tablet_manager.cc:1428] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:00.051337 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.050895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:54979)
0927 10:49:00.051170 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.051233 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
I20260927 10:49:00.052062 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.051728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:36027)
0927 10:49:00.051938 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.051988 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260927 10:49:00.053406 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.052969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:60949)
0927 10:49:00.053209 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.053307 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":92,"threads_started":1}
I20260927 10:49:00.053742 16720 heartbeater.cc:503] Master 127.9.39.189:38161 was elected leader, sending a full tablet report...
I20260927 10:49:00.053769 16446 heartbeater.cc:503] Master 127.9.39.189:38161 was elected leader, sending a full tablet report...
I20260927 10:49:00.054039 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.050438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:45567)
0927 10:49:00.053865 (+ 3427us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.053922 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3367,"thread_start_us":89,"threads_started":1}
I20260927 10:49:00.054803 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.054430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:38703)
0927 10:49:00.054644 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.054686 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260927 10:49:00.055534 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.054885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:35513)
0927 10:49:00.055420 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.055472 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461}
I20260927 10:49:00.056325 16583 heartbeater.cc:503] Master 127.9.39.189:38161 was elected leader, sending a full tablet report...
W20260927 10:49:00.066031 16584 tablet.cc:2369] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:49:00.112556 16832 raft_consensus.cc:482] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:00.112705 16832 raft_consensus.cc:504] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.113133 16832 leader_election.cc:288] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767), abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:49:00.113683 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31736a4cc8a140aa9813956a5a51a8bd" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5efe62e66f6046968b0fc59352b2b122" is_pre_election: true
I20260927 10:49:00.113729 16535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31736a4cc8a140aa9813956a5a51a8bd" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe31483246640728ab67024a8b71630" is_pre_election: true
I20260927 10:49:00.113860 16672 raft_consensus.cc:2454] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 0.
I20260927 10:49:00.113860 16535 raft_consensus.cc:2454] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 0.
I20260927 10:49:00.114202 16334 leader_election.cc:302] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe31483246640728ab67024a8b71630, fd60eeebf2084b8baf3617e842cdd7ac; no voters:
I20260927 10:49:00.114403 16832 raft_consensus.cc:2781] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:00.114459 16832 raft_consensus.cc:482] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:00.114507 16832 raft_consensus.cc:3037] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:00.115772 16832 raft_consensus.cc:504] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.116124 16832 leader_election.cc:288] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 election: Requested vote from peers 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767), abe31483246640728ab67024a8b71630 (127.9.39.130:41315)
I20260927 10:49:00.116493 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31736a4cc8a140aa9813956a5a51a8bd" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5efe62e66f6046968b0fc59352b2b122"
I20260927 10:49:00.116514 16535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31736a4cc8a140aa9813956a5a51a8bd" candidate_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe31483246640728ab67024a8b71630"
I20260927 10:49:00.116627 16672 raft_consensus.cc:3037] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:00.116627 16535 raft_consensus.cc:3037] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:00.117741 16672 raft_consensus.cc:2454] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 1.
I20260927 10:49:00.117743 16535 raft_consensus.cc:2454] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd60eeebf2084b8baf3617e842cdd7ac in term 1.
I20260927 10:49:00.118017 16334 leader_election.cc:302] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5efe62e66f6046968b0fc59352b2b122, fd60eeebf2084b8baf3617e842cdd7ac; no voters:
I20260927 10:49:00.118175 16832 raft_consensus.cc:2781] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:00.118402 16832 raft_consensus.cc:686] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 1 LEADER]: Becoming Leader. State: Replica: fd60eeebf2084b8baf3617e842cdd7ac, State: Running, Role: LEADER
I20260927 10:49:00.118567 16832 consensus_queue.cc:237] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } }
I20260927 10:49:00.119140 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.118880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:42307)
0927 10:49:00.118977 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.119038 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 10:49:00.120333 16073 catalog_manager.cc:5495] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac reported cstate change: term changed from 0 to 1, leader changed from <none> to fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129). New cstate: current_term: 1 leader_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd60eeebf2084b8baf3617e842cdd7ac" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 } health_report { overall_health: UNKNOWN } } }
I20260927 10:49:00.143316 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.140200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.9.39.129:33189)
0927 10:49:00.140353 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:00.140356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:00.140370 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:49:00.140413 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:00.140415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:00.140458 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:00.140519 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:00.140795 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.141339 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.142364 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:00.142572 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:00.142650 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:00.142930 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:00.142995 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:00.143050 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:00.143156 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 10:49:00.143208 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 10:49:00.145506 16535 raft_consensus.cc:1261] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Refusing update from remote peer fd60eeebf2084b8baf3617e842cdd7ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:00.145588 16672 raft_consensus.cc:1261] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Refusing update from remote peer fd60eeebf2084b8baf3617e842cdd7ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:00.146021 16838 consensus_queue.cc:1035] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe31483246640728ab67024a8b71630" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:00.146196 16832 consensus_queue.cc:1035] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5efe62e66f6046968b0fc59352b2b122" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:00.147711 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.147373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:55613)
0927 10:49:00.147535 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.147603 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260927 10:49:00.148038 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.147777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:52325)
0927 10:49:00.147926 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.147983 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 10:49:00.150995 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.150725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:37353)
0927 10:49:00.150874 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.150929 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 10:49:00.151450 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.151170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.130:45519)
0927 10:49:00.151331 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.151386 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260927 10:49:00.153621 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.153387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.129:59431)
0927 10:49:00.153511 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.153563 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260927 10:49:00.154381 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.154002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:38161 (local address 127.9.39.131:54927)
0927 10:49:00.154261 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.154314 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260927 10:49:00.155403 16072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:00.155833 16072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:00.159731 16072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5bad59ad42147f898036f6467d37c28: Sending DeleteTablet for 3 replicas of tablet 31736a4cc8a140aa9813956a5a51a8bd
I20260927 10:49:00.160394 16652 tablet_service.cc:1514] Processing DeleteTablet for tablet 31736a4cc8a140aa9813956a5a51a8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:00 UTC) from {username='oryx', principal='oryx/127.9.39.189@KRBTEST.COM'} at 127.0.0.1:47856
I20260927 10:49:00.160524 16515 tablet_service.cc:1514] Processing DeleteTablet for tablet 31736a4cc8a140aa9813956a5a51a8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:00 UTC) from {username='oryx', principal='oryx/127.9.39.189@KRBTEST.COM'} at 127.0.0.1:45456
I20260927 10:49:00.160627 16378 tablet_service.cc:1514] Processing DeleteTablet for tablet 31736a4cc8a140aa9813956a5a51a8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:00 UTC) from {username='oryx', principal='oryx/127.9.39.189@KRBTEST.COM'} at 127.0.0.1:35590
I20260927 10:49:00.160835 16851 tablet_replica.cc:330] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: stopping tablet replica
I20260927 10:49:00.160950 16852 tablet_replica.cc:330] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac: stopping tablet replica
I20260927 10:49:00.160952 16851 raft_consensus.cc:2229] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:00.161036 16850 tablet_replica.cc:330] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: stopping tablet replica
I20260927 10:49:00.161150 16852 raft_consensus.cc:2229] T 31736a4cc8a140aa9813956a5a51a8bd P fd60eeebf2084b8baf3617e842cdd7ac [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:49:00.161185 16850 raft_consensus.cc:2229] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:00.161213 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 16218
I20260927 10:49:00.161908 16851 raft_consensus.cc:2258] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:00.162607 16851 ts_tablet_manager.cc:1905] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:00.164685 16851 ts_tablet_manager.cc:1918] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:00.164777 16851 log.cc:1196] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/31736a4cc8a140aa9813956a5a51a8bd
I20260927 10:49:00.165017 16851 ts_tablet_manager.cc:1939] T 31736a4cc8a140aa9813956a5a51a8bd P abe31483246640728ab67024a8b71630: Deleting consensus metadata
I20260927 10:49:00.165088 16850 raft_consensus.cc:2258] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:00.166016 16060 catalog_manager.cc:4841] TS abe31483246640728ab67024a8b71630 (127.9.39.130:41315): tablet 31736a4cc8a140aa9813956a5a51a8bd (table test-table [id=48c72823a0ae42fe9cbc6b3a1544b27c]) successfully deleted
I20260927 10:49:00.166251 16850 ts_tablet_manager.cc:1905] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:00.169390 16850 ts_tablet_manager.cc:1918] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:00.169474 16850 log.cc:1196] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/31736a4cc8a140aa9813956a5a51a8bd
I20260927 10:49:00.169739 16850 ts_tablet_manager.cc:1939] T 31736a4cc8a140aa9813956a5a51a8bd P 5efe62e66f6046968b0fc59352b2b122: Deleting consensus metadata
I20260927 10:49:00.170468 16060 catalog_manager.cc:4841] TS 5efe62e66f6046968b0fc59352b2b122 (127.9.39.131:44767): tablet 31736a4cc8a140aa9813956a5a51a8bd (table test-table [id=48c72823a0ae42fe9cbc6b3a1544b27c]) successfully deleted
I20260927 10:49:00.172070 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 16452
I20260927 10:49:00.172857 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:00.172288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:33189 (local address 127.0.0.1:35612)
0927 10:49:00.172688 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:00.172760 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
W20260927 10:49:00.173293 16060 catalog_manager.cc:4584] TS fd60eeebf2084b8baf3617e842cdd7ac (127.9.39.129:33189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31736a4cc8a140aa9813956a5a51a8bd: Network error: Client connection negotiation failed: client connection to 127.9.39.129:33189: connect: Connection refused (error 111)
I20260927 10:49:00.177812 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 16588
I20260927 10:49:00.183485 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 15937
I20260927 10:49:00.189265 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 16045
2026-09-27T10:49:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7222 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:49:00 dist-test-slave-p9q6 krb5kdc[16855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:49:00 dist-test-slave-p9q6 krb5kdc[16855](info): set up 2 sockets
Sep 27 10:49:00 dist-test-slave-p9q6 krb5kdc[16855](info): commencing operation
krb5kdc: starting...
W20260927 10:49:01.275281 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:49:01 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab.
Entry for principal oryx/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab.
I20260927 10:49:01.397953 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37331
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab
--principal=oryx/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:37331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:01.507714 16868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:01.507998 16868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:01.508037 16868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:01.512140 16868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:01.512207 16868 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 10:49:01.512224 16868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:01.512243 16868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:01.512290 16868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:01.512328 16868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:01.517618 16868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:37331
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.190.keytab
--principal=oryx/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:01.518687 16868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:01.519740 16868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:01.528883 16873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:01.529028 16868 server_base.cc:1034] running on GCE node
W20260927 10:49:01.528877 16874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:01.528848 16876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:01.529922 16868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:01.531168 16868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:01.532380 16868 hybrid_clock.cc:648] HybridClock initialized: now 1790506141532343 us; error 49 us; skew 500 ppm
Sep 27 10:49:01 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:01.535328 16868 init.cc:377] Logged in from keytab as oryx/127.9.39.190@KRBTEST.COM (short username oryx)
I20260927 10:49:01.536294 16868 webserver.cc:460] Webserver started at http://127.9.39.190:44971/ using document root <none> and password file <none>
I20260927 10:49:01.536854 16868 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:01.536918 16868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:01.537092 16868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:01.539327 16868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "43bb8a518fe4483bab5a27063e87c1a3"
format_stamp: "Formatted at 2026-09-27 10:49:01 on dist-test-slave-p9q6"
server_key: "c261f06bb8b12234db4e06624aebaf8a"
server_key_iv: "359fe86100c844c8c3107b2b0fa0b474"
server_key_version: "encryptionkey@0"
I20260927 10:49:01.539902 16868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "43bb8a518fe4483bab5a27063e87c1a3"
format_stamp: "Formatted at 2026-09-27 10:49:01 on dist-test-slave-p9q6"
server_key: "c261f06bb8b12234db4e06624aebaf8a"
server_key_iv: "359fe86100c844c8c3107b2b0fa0b474"
server_key_version: "encryptionkey@0"
I20260927 10:49:01.543989 16868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:49:01.546535 16883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:01.547885 16868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:01.547997 16868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "43bb8a518fe4483bab5a27063e87c1a3"
format_stamp: "Formatted at 2026-09-27 10:49:01 on dist-test-slave-p9q6"
server_key: "c261f06bb8b12234db4e06624aebaf8a"
server_key_iv: "359fe86100c844c8c3107b2b0fa0b474"
server_key_version: "encryptionkey@0"
I20260927 10:49:01.548089 16868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:01.561072 16868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:01.564052 16868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:01.564203 16868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:01.572095 16868 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:37331
I20260927 10:49:01.572113 16945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:37331 every 8 connection(s)
I20260927 10:49:01.573041 16868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:49:01.574091 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 16868
I20260927 10:49:01.574205 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:49:01.574496 9374 external_mini_cluster.cc:1428] Setting key e84bda41929b081ef1642c4860c185a0
I20260927 10:49:01.576330 16946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:01.583665 16946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Bootstrap starting.
Sep 27 10:49:01 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:01.586570 16946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:01.588044 16946 log.cc:824] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:01.590538 16946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: No bootstrap required, opened a new log
I20260927 10:49:01.591135 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:01.575983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.9.39.190:37331)
0927 10:49:01.576422 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:01.576442 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:01.576484 (+ 42us) server_negotiation.cc:407] Connection header received
0927 10:49:01.577368 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:01.577392 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:01.577826 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:01.578222 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:01.578962 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.580648 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.581546 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.582293 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.582472 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:01.585403 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:01.585409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:01.585425 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:01.585467 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:01.587731 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:01.588186 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:01.588191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:01.588197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:01.588338 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:01.588713 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:01.588715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:01.588722 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:01.589304 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:01.589503 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:01.589763 (+ 260us) server_negotiation.cc:299] Negotiation successful
0927 10:49:01.590080 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":120,"threads_started":1}
I20260927 10:49:01.594646 16946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bb8a518fe4483bab5a27063e87c1a3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37331 } }
I20260927 10:49:01.594965 16946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:01.595031 16946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43bb8a518fe4483bab5a27063e87c1a3, State: Initialized, Role: FOLLOWER
I20260927 10:49:01.595464 16946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bb8a518fe4483bab5a27063e87c1a3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37331 } }
I20260927 10:49:01.595695 16946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:49:01.595793 16946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:49:01.595909 16946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:01.596909 16946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bb8a518fe4483bab5a27063e87c1a3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37331 } }
I20260927 10:49:01.597276 16946 leader_election.cc:302] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43bb8a518fe4483bab5a27063e87c1a3; no voters:
I20260927 10:49:01.597646 16946 leader_election.cc:288] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:49:01.597808 16951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:01.598055 16951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [term 1 LEADER]: Becoming Leader. State: Replica: 43bb8a518fe4483bab5a27063e87c1a3, State: Running, Role: LEADER
I20260927 10:49:01.598344 16951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bb8a518fe4483bab5a27063e87c1a3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37331 } }
I20260927 10:49:01.598970 16946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:49:01.600095 16953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43bb8a518fe4483bab5a27063e87c1a3. Latest consensus state: current_term: 1 leader_uuid: "43bb8a518fe4483bab5a27063e87c1a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bb8a518fe4483bab5a27063e87c1a3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37331 } } }
I20260927 10:49:01.600234 16953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:01.601096 16952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43bb8a518fe4483bab5a27063e87c1a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bb8a518fe4483bab5a27063e87c1a3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37331 } } }
I20260927 10:49:01.601197 16952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3 [sys.catalog]: This master's current role is: LEADER
W20260927 10:49:01.603971 16966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:49:01.604055 16966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:49:01.604125 16960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:49:01.604990 16960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:49:01.610740 16960 catalog_manager.cc:1347] Generated new cluster ID: b98b1fd47d8e4efe9a8f9c398cb12dd4
I20260927 10:49:01.610833 16960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:49:01.624886 16960 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:49:01.626108 16960 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:49:01.631922 16960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Generated new TSK 0
I20260927 10:49:01.632704 16960 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab.
Entry for principal oryx/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab.
I20260927 10:49:01.662549 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37331
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:01.773703 16972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:01.773977 16972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:01.774049 16972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:01.778601 16972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:01.778689 16972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:01.778777 16972 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:49:01.778863 16972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:49:01.784111 16972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:37331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:01.785140 16972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:01.786162 16972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:01.795591 16978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:01.795600 16977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:01.795631 16980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:01.795871 16972 server_base.cc:1034] running on GCE node
I20260927 10:49:01.796254 16972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:01.796872 16972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:01.798050 16972 hybrid_clock.cc:648] HybridClock initialized: now 1790506141798034 us; error 27 us; skew 500 ppm
Sep 27 10:49:01 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:01.801064 16972 init.cc:377] Logged in from keytab as oryx/127.9.39.129@KRBTEST.COM (short username oryx)
I20260927 10:49:01.801970 16972 webserver.cc:460] Webserver started at http://127.9.39.129:46177/ using document root <none> and password file <none>
I20260927 10:49:01.802564 16972 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:01.802649 16972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:01.802876 16972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:01.804867 16972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "71b9d51a93e4426dbfb2dadb8d21771d"
format_stamp: "Formatted at 2026-09-27 10:49:01 on dist-test-slave-p9q6"
server_key: "62463a09ba16c0422a39d46a651e44e6"
server_key_iv: "6aa1e1eb97c49357f983d0fa05d8c376"
server_key_version: "encryptionkey@0"
I20260927 10:49:01.805373 16972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "71b9d51a93e4426dbfb2dadb8d21771d"
format_stamp: "Formatted at 2026-09-27 10:49:01 on dist-test-slave-p9q6"
server_key: "62463a09ba16c0422a39d46a651e44e6"
server_key_iv: "6aa1e1eb97c49357f983d0fa05d8c376"
server_key_version: "encryptionkey@0"
I20260927 10:49:01.809088 16972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 10:49:01.811393 16987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:01.812569 16972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:01.812660 16972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "71b9d51a93e4426dbfb2dadb8d21771d"
format_stamp: "Formatted at 2026-09-27 10:49:01 on dist-test-slave-p9q6"
server_key: "62463a09ba16c0422a39d46a651e44e6"
server_key_iv: "6aa1e1eb97c49357f983d0fa05d8c376"
server_key_version: "encryptionkey@0"
I20260927 10:49:01.812737 16972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:01.835405 16972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:01.838581 16972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:01.838809 16972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:01.840230 16972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:01.840279 16972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:01.840319 16972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:01.840335 16972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:49:01 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:01.851229 16972 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:37443
I20260927 10:49:01.852334 16972 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:49:01.852345 17102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:37443 every 8 connection(s)
I20260927 10:49:01.854283 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:01.841009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:41407 (local address 127.9.39.190:37331)
0927 10:49:01.841202 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:01.841209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:01.842003 (+ 794us) server_negotiation.cc:407] Connection header received
0927 10:49:01.843200 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:01.843207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:01.843263 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:01.843345 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:01.844990 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.845627 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.846842 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.847272 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.847405 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:01.850409 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:01.850412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:01.850414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:01.850439 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:01.852195 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:01.852801 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:01.852805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:01.852807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:01.852904 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:01.853317 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:01.853320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:01.853322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:01.853522 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:01.853603 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:01.854036 (+ 433us) server_negotiation.cc:299] Negotiation successful
0927 10:49:01.854133 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 10:49:01.855149 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:01.841404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.9.39.129:41407)
0927 10:49:01.841844 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:01.841892 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:01.842931 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:01.843786 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:01.843797 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:01.844166 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:01.844836 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:01.844853 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.845765 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.845768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:01.846654 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:01.846663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.847453 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.847458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:01.847670 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:01.848275 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:01.848294 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:01.850201 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:01.852325 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:01.852331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:01.852345 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:01.852683 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:01.853021 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:01.853024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:01.853026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:01.853198 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:01.853618 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:01.853622 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:01.853835 (+ 213us) client_negotiation.cc:769] Sending connection context
0927 10:49:01.853990 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 10:49:01.854247 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
I20260927 10:49:01.856323 17103 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37331
I20260927 10:49:01.856456 17103 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:01.857034 17103 heartbeater.cc:511] Master 127.9.39.190:37331 requested a full tablet report, sending...
I20260927 10:49:01.858806 16899 ts_manager.cc:194] Registered new tserver with Master: 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443)
I20260927 10:49:01.859275 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 16972
I20260927 10:49:01.859431 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:49:01.859699 9374 external_mini_cluster.cc:1428] Setting key 486c1023903cea680013fe404f346ecc
I20260927 10:49:01.860363 16899 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:41407
I20260927 10:49:01.866312 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:01.858801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:32937 (local address 127.9.39.190:37331)
0927 10:49:01.858937 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:01.858941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:01.859114 (+ 173us) server_negotiation.cc:407] Connection header received
0927 10:49:01.859239 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:01.859243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:01.859295 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:01.859406 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:01.859890 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.860443 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.861440 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.861829 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.861955 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:01.863145 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:01.863151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:01.863156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:01.863194 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:01.864621 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:01.865105 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:01.865107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:01.865109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:01.865212 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:01.865528 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:01.865533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:01.865535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:01.865723 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:01.865802 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:01.866109 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 10:49:01.866206 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:49:01.866501 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:01.858658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.9.39.129:32937)
0927 10:49:01.858995 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:01.859010 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:01.859136 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:01.859434 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:01.859436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:01.859645 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:01.859759 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:01.859766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.860571 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.860574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:01.861308 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:01.861327 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:01.861972 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:01.861976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:01.862065 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:01.862628 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:01.862642 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:01.863011 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:01.864731 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:01.864738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:01.864743 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:01.865012 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:01.865305 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:01.865310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:01.865312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:01.865416 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:01.865829 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:01.865834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:01.865946 (+ 112us) client_negotiation.cc:769] Sending connection context
0927 10:49:01.866061 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 10:49:01.866233 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab.
Entry for principal oryx/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab.
I20260927 10:49:01.886091 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37331
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:01.993700 17112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:01.993929 17112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:01.993999 17112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:01.997993 17112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:01.998052 17112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:01.998090 17112 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:49:01.998124 17112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:49:02.003671 17112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:37331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:02.004683 17112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:02.005707 17112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:02.015199 17118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:02.015275 17117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:02.015199 17120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:02.015489 17112 server_base.cc:1034] running on GCE node
I20260927 10:49:02.016006 17112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:02.016729 17112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:02.017932 17112 hybrid_clock.cc:648] HybridClock initialized: now 1790506142017895 us; error 51 us; skew 500 ppm
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:02.020670 17112 init.cc:377] Logged in from keytab as oryx/127.9.39.130@KRBTEST.COM (short username oryx)
I20260927 10:49:02.021528 17112 webserver.cc:460] Webserver started at http://127.9.39.130:44297/ using document root <none> and password file <none>
I20260927 10:49:02.022038 17112 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:02.022094 17112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:02.022253 17112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:02.024271 17112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "ca6c14a2977b4a1f890beabf31b3c584"
format_stamp: "Formatted at 2026-09-27 10:49:02 on dist-test-slave-p9q6"
server_key: "62105bebdf5cc2e2bc6978e8a2c39ccd"
server_key_iv: "cf6a85ad62b47ee433f346364f440c42"
server_key_version: "encryptionkey@0"
I20260927 10:49:02.024766 17112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "ca6c14a2977b4a1f890beabf31b3c584"
format_stamp: "Formatted at 2026-09-27 10:49:02 on dist-test-slave-p9q6"
server_key: "62105bebdf5cc2e2bc6978e8a2c39ccd"
server_key_iv: "cf6a85ad62b47ee433f346364f440c42"
server_key_version: "encryptionkey@0"
I20260927 10:49:02.028362 17112 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260927 10:49:02.030650 17127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:02.031859 17112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:49:02.031950 17112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "ca6c14a2977b4a1f890beabf31b3c584"
format_stamp: "Formatted at 2026-09-27 10:49:02 on dist-test-slave-p9q6"
server_key: "62105bebdf5cc2e2bc6978e8a2c39ccd"
server_key_iv: "cf6a85ad62b47ee433f346364f440c42"
server_key_version: "encryptionkey@0"
I20260927 10:49:02.032078 17112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:02.055850 17112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:02.059077 17112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:02.059288 17112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:02.060730 17112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:02.060804 17112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:02.060887 17112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:02.060937 17112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:02.072188 17112 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:35923
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:02.073330 17112 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:49:02.074132 17242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:35923 every 8 connection(s)
I20260927 10:49:02.076910 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.061887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:47415 (local address 127.9.39.190:37331)
0927 10:49:02.062081 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.062085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.063121 (+ 1036us) server_negotiation.cc:407] Connection header received
0927 10:49:02.064319 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.064323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.064382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.064497 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.066274 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.067252 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.069166 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.069388 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.069476 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.073054 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:02.073057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:02.073059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:02.073086 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:02.074928 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.075479 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.075482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.075483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.075562 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.075905 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.075907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.075908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.076091 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:02.076200 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.076682 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.076784 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 10:49:02.077736 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.062428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.9.39.130:47415)
0927 10:49:02.062972 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.063014 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.064030 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.064721 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.064732 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.065273 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:02.066060 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.066084 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.067441 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.067445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.069022 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.069032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.069560 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.069566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.069855 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.070627 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:02.070661 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:02.072827 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:02.075041 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.075046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.075057 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.075373 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.075674 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.075677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.075679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.075819 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.076224 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:02.076230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:02.076462 (+ 232us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.076629 (+ 167us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.076882 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":176,"threads_started":1}
I20260927 10:49:02.079144 17243 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37331
I20260927 10:49:02.079312 17243 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:02.079809 17243 heartbeater.cc:511] Master 127.9.39.190:37331 requested a full tablet report, sending...
I20260927 10:49:02.080914 16900 ts_manager.cc:194] Registered new tserver with Master: ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923)
I20260927 10:49:02.081789 16900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:47415
I20260927 10:49:02.082574 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 17112
I20260927 10:49:02.082744 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:49:02.083056 9374 external_mini_cluster.cc:1428] Setting key 483a71c1f576e8c8964352c288e9b6e7
I20260927 10:49:02.089207 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.081310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:56799 (local address 127.9.39.190:37331)
0927 10:49:02.081429 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.081433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.081539 (+ 106us) server_negotiation.cc:407] Connection header received
0927 10:49:02.081658 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.081660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.081693 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.081755 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.082235 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.082928 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.083989 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.084261 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.084390 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.085534 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:02.085538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:02.085541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:02.085563 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:02.087283 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.087835 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.087837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.087838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.087920 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.088245 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.088248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.088258 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.088460 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:02.088546 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.088921 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.089049 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:49:02.089269 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.081178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.9.39.130:56799)
0927 10:49:02.081470 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.081485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.081581 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.081799 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.081805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.082013 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:02.082153 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.082159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.083047 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.083051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.083866 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.083877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.084392 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.084396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.084490 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.085020 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:02.085031 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:02.085405 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:02.087432 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.087439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.087444 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.087725 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.088015 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.088021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.088026 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.088129 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.088566 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:02.088572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:02.088702 (+ 130us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.088824 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.088995 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab.
Entry for principal oryx/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab.
I20260927 10:49:02.109869 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:37331
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:02.220752 17252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:02.221006 17252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:02.221050 17252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:02.224995 17252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:02.225056 17252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:02.225095 17252 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 10:49:02.225127 17252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:49:02.230355 17252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.39.190:37331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:02.231374 17252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:02.232343 17252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:02.242017 17258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:02.242033 17260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:02.242085 17257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:02.242271 17252 server_base.cc:1034] running on GCE node
I20260927 10:49:02.242856 17252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:02.243502 17252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:02.244657 17252 hybrid_clock.cc:648] HybridClock initialized: now 1790506142244623 us; error 46 us; skew 500 ppm
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:02.247342 17252 init.cc:377] Logged in from keytab as oryx/127.9.39.131@KRBTEST.COM (short username oryx)
I20260927 10:49:02.248176 17252 webserver.cc:460] Webserver started at http://127.9.39.131:43913/ using document root <none> and password file <none>
I20260927 10:49:02.248672 17252 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:02.248730 17252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:02.248888 17252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:02.250913 17252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "82f87b2e40154883901485d4b8f7d7e0"
format_stamp: "Formatted at 2026-09-27 10:49:02 on dist-test-slave-p9q6"
server_key: "3f6f41ca11431f683c70a4f76edff28c"
server_key_iv: "20427e34087b50e9ca032fb9ed066b7d"
server_key_version: "encryptionkey@0"
I20260927 10:49:02.251456 17252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "82f87b2e40154883901485d4b8f7d7e0"
format_stamp: "Formatted at 2026-09-27 10:49:02 on dist-test-slave-p9q6"
server_key: "3f6f41ca11431f683c70a4f76edff28c"
server_key_iv: "20427e34087b50e9ca032fb9ed066b7d"
server_key_version: "encryptionkey@0"
I20260927 10:49:02.254992 17252 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 10:49:02.257364 17267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:02.258550 17252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:02.258667 17252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "82f87b2e40154883901485d4b8f7d7e0"
format_stamp: "Formatted at 2026-09-27 10:49:02 on dist-test-slave-p9q6"
server_key: "3f6f41ca11431f683c70a4f76edff28c"
server_key_iv: "20427e34087b50e9ca032fb9ed066b7d"
server_key_version: "encryptionkey@0"
I20260927 10:49:02.258795 17252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:02.276932 17252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:02.280143 17252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:02.280346 17252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:02.281704 17252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:02.281773 17252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:02.281844 17252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:02.281885 17252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:02.292070 17252 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:33629
I20260927 10:49:02.292140 17382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:33629 every 8 connection(s)
I20260927 10:49:02.293148 17252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:49:02.295855 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.282530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:33281 (local address 127.9.39.190:37331)
0927 10:49:02.282713 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.282720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.283548 (+ 828us) server_negotiation.cc:407] Connection header received
0927 10:49:02.284744 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.284750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.284821 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.284932 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.286417 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.287329 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.289122 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.289473 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.289612 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.292480 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:02.292484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:02.292487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:02.292513 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:02.294096 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.294549 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.294551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.294552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.294623 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.294949 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.294952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.294953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.295114 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:02.295233 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.295650 (+ 417us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.295742 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:49:02.296164 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 17252
I20260927 10:49:02.296306 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:49:02.296576 9374 external_mini_cluster.cc:1428] Setting key 15456be03b693542165a8edd44f5d8a6
I20260927 10:49:02.296720 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.282944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.9.39.131:33281)
0927 10:49:02.283386 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.283435 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.284451 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.285135 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.285143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.285543 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:02.286251 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.286269 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.287478 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.287481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.288971 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.288981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.289624 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.289626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.289761 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.290445 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:02.290467 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:02.292328 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:02.294216 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.294221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.294230 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.294464 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.294732 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.294738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.294742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.294869 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.295243 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:02.295248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:02.295444 (+ 196us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.295585 (+ 141us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.295821 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":137,"threads_started":1}
I20260927 10:49:02.297932 17383 heartbeater.cc:347] Connected to a master server at 127.9.39.190:37331
I20260927 10:49:02.298074 17383 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:02.298614 17383 heartbeater.cc:511] Master 127.9.39.190:37331 requested a full tablet report, sending...
I20260927 10:49:02.299741 16898 ts_manager.cc:194] Registered new tserver with Master: 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:02.300581 16898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:33281
I20260927 10:49:02.307662 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.300088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:35161 (local address 127.9.39.190:37331)
0927 10:49:02.300202 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.300211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.300406 (+ 195us) server_negotiation.cc:407] Connection header received
0927 10:49:02.300567 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.300570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.300608 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.300713 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.302291 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.302851 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.303636 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.303871 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.303977 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.304920 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:02.304924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:02.304926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:02.304948 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:02.306072 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.306483 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.306488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.306490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.306570 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.306874 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.306879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.306881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.307043 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:02.307177 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.307410 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.307510 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 10:49:02.307698 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.300026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.9.39.131:35161)
0927 10:49:02.300346 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.300360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.300449 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.300741 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.300744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.300939 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:02.301026 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.301031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.302970 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.302973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.303518 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.303525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.303980 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.303984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.304076 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.304471 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:02.304481 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:02.304791 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:02.306188 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.306195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.306197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.306386 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.306664 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.306667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.306669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.306785 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.307145 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:02.307148 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:02.307236 (+ 88us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.307357 (+ 121us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.307487 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":94,"threads_started":1}
I20260927 10:49:02.310563 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 10:49:02.333987 16949 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:02.327789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55578 (local address 127.9.39.190:37331)
0927 10:49:02.327932 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.327935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.327947 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:49:02.327988 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.327990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.328031 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.328128 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.328905 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.329447 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.330345 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.330553 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.330675 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.333810 (+ 3135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55578: BlockingRecv error: recv got EOF from 127.0.0.1:55578 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:02.343631 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.335640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55580 (local address 127.9.39.190:37331)
0927 10:49:02.335784 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.335788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.335861 (+ 73us) server_negotiation.cc:407] Connection header received
0927 10:49:02.335935 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.335937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.335972 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.336085 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.336738 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.337213 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.337980 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.338190 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.338271 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.340606 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:02.340609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:02.340615 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:02.340636 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:02.342055 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.342470 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.342473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.342474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.342551 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.342859 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.342861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.342863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.343028 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:02.343132 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.343333 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.343421 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 10:49:02.346766 16898 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:49:02.351786 16898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:02.367522 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.361895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.9.39.129:37443)
0927 10:49:02.362962 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.362970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.362995 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:49:02.363069 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.363073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.363219 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.363379 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.363719 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.364499 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.366025 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.366765 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.366871 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.366978 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.367294 (+ 316us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.367375 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":97,"threads_started":1}
I20260927 10:49:02.367522 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.361340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37443 (local address 127.0.0.1:46040)
0927 10:49:02.362270 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.362286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.362369 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.363498 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.363503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.363523 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.363618 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.363622 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.364625 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.364628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.365882 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.365893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.367029 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.367032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.367109 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.367135 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.367252 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.367374 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":66,"thread_start_us":106,"threads_started":1}
I20260927 10:49:02.367668 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.361845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33629 (local address 127.0.0.1:34656)
0927 10:49:02.362601 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.362612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.362746 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.363359 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.363362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.363379 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.363459 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.363467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.364783 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.364786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.365698 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.365706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.367054 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.367057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.367242 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.367299 (+ 57us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.367368 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.367548 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":211,"thread_start_us":90,"threads_started":1}
I20260927 10:49:02.367728 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.362278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.9.39.131:33629)
0927 10:49:02.362904 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.362915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.362939 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:49:02.363021 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.363025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.363191 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.363335 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.363651 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.364661 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.365823 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.366639 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.366833 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.366999 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.367506 (+ 507us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.367596 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":106,"threads_started":1}
I20260927 10:49:02.367897 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.361143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35923 (local address 127.0.0.1:40852)
0927 10:49:02.361673 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.361723 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.361856 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.362747 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.362753 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.362775 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.362916 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.362923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.364355 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.364358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.365882 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.365893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.367054 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.367057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.367245 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.367297 (+ 52us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.367368 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.367476 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":69,"thread_start_us":64,"threads_started":1}
I20260927 10:49:02.368745 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.361313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.9.39.130:35923)
0927 10:49:02.362229 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.362241 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.362264 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:49:02.362341 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.362348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.362574 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.362752 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.363025 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.364234 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.365999 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.366761 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.366848 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.366999 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.368518 (+ 1519us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.368604 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":119,"threads_started":1}
I20260927 10:49:02.370301 17317 tablet_service.cc:1467] Processing CreateTablet for tablet b9c8559ae16f485fb58f5117baa79d50 (DEFAULT_TABLE table=test-table [id=8a81c555cacf473cb9bd7f76965ab0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:02.370301 17037 tablet_service.cc:1467] Processing CreateTablet for tablet b9c8559ae16f485fb58f5117baa79d50 (DEFAULT_TABLE table=test-table [id=8a81c555cacf473cb9bd7f76965ab0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:02.371304 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c8559ae16f485fb58f5117baa79d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.371320 17317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c8559ae16f485fb58f5117baa79d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.373121 17177 tablet_service.cc:1467] Processing CreateTablet for tablet b9c8559ae16f485fb58f5117baa79d50 (DEFAULT_TABLE table=test-table [id=8a81c555cacf473cb9bd7f76965ab0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:02.374008 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c8559ae16f485fb58f5117baa79d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.376637 17411 tablet_bootstrap.cc:492] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Bootstrap starting.
I20260927 10:49:02.378844 17411 tablet_bootstrap.cc:654] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.379575 17412 tablet_bootstrap.cc:492] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Bootstrap starting.
I20260927 10:49:02.379588 17411 log.cc:824] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:02.381301 17411 tablet_bootstrap.cc:492] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: No bootstrap required, opened a new log
I20260927 10:49:02.381337 17414 tablet_bootstrap.cc:492] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Bootstrap starting.
I20260927 10:49:02.381589 17411 ts_tablet_manager.cc:1397] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260927 10:49:02.381635 17412 tablet_bootstrap.cc:654] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.382385 17412 log.cc:824] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:02.383533 17414 tablet_bootstrap.cc:654] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.384102 17412 tablet_bootstrap.cc:492] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: No bootstrap required, opened a new log
I20260927 10:49:02.384297 17412 ts_tablet_manager.cc:1397] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260927 10:49:02.384320 17414 log.cc:824] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:02.385828 17411 raft_consensus.cc:348] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.386102 17411 raft_consensus.cc:374] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.386123 17414 tablet_bootstrap.cc:492] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: No bootstrap required, opened a new log
I20260927 10:49:02.386193 17411 raft_consensus.cc:729] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.386283 17414 ts_tablet_manager.cc:1397] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:49:02.386643 17411 consensus_queue.cc:260] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.387288 17103 heartbeater.cc:503] Master 127.9.39.190:37331 was elected leader, sending a full tablet report...
I20260927 10:49:02.387631 17412 raft_consensus.cc:348] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.387843 17412 raft_consensus.cc:374] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.387912 17412 raft_consensus.cc:729] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.388325 17412 consensus_queue.cc:260] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.389442 17412 ts_tablet_manager.cc:1428] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260927 10:49:02.389878 17414 raft_consensus.cc:348] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.390103 17383 heartbeater.cc:503] Master 127.9.39.190:37331 was elected leader, sending a full tablet report...
I20260927 10:49:02.390170 17414 raft_consensus.cc:374] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.390226 17414 raft_consensus.cc:729] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c14a2977b4a1f890beabf31b3c584, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.390686 17414 consensus_queue.cc:260] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.391332 17243 heartbeater.cc:503] Master 127.9.39.190:37331 was elected leader, sending a full tablet report...
I20260927 10:49:02.391583 17414 ts_tablet_manager.cc:1428] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:49:02.392302 17411 ts_tablet_manager.cc:1428] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260927 10:49:02.519086 17418 raft_consensus.cc:482] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:02.519335 17418 raft_consensus.cc:504] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.520526 17418 leader_election.cc:288] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923), 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443)
I20260927 10:49:02.524338 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.520723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35923 (local address 127.9.39.131:36185)
0927 10:49:02.520905 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.520921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.521013 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.521361 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.521367 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.521386 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.521496 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.521501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.522302 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.522304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.523330 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.523341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.524011 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.524014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.524089 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.524109 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.524176 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.524232 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260927 10:49:02.524401 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.520868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:36185 (local address 127.9.39.130:35923)
0927 10:49:02.520988 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.520993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.521006 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:02.521148 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.521151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.521224 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.521326 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.521634 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.522191 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.523468 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.523876 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.524023 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.524077 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.524208 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.524276 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:49:02.525002 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.520805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37443 (local address 127.9.39.131:51901)
0927 10:49:02.521195 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.521206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.521287 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.521574 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.521579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.521600 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.522038 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.522044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.522944 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.522947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.523835 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.523843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.524676 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.524679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.524781 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.524798 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.524858 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.524906 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":115,"threads_started":1}
I20260927 10:49:02.525091 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.520901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:51901 (local address 127.9.39.129:37443)
0927 10:49:02.521025 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.521029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.521297 (+ 268us) server_negotiation.cc:407] Connection header received
0927 10:49:02.521414 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.521419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.521473 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.521534 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.522157 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.522822 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.523956 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.524553 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.524666 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.524710 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.524897 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.524976 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:49:02.525038 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c8559ae16f485fb58f5117baa79d50" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584" is_pre_election: true
I20260927 10:49:02.525398 17197 raft_consensus.cc:2454] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 0.
I20260927 10:49:02.525643 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c8559ae16f485fb58f5117baa79d50" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" is_pre_election: true
I20260927 10:49:02.525956 17057 raft_consensus.cc:2454] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 0.
I20260927 10:49:02.525971 17269 leader_election.cc:302] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82f87b2e40154883901485d4b8f7d7e0, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:02.526275 17418 raft_consensus.cc:2781] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:02.526396 17418 raft_consensus.cc:482] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:02.526463 17418 raft_consensus.cc:3037] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.527527 17418 raft_consensus.cc:504] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.527923 17418 leader_election.cc:288] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 election: Requested vote from peers ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923), 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443)
I20260927 10:49:02.528251 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c8559ae16f485fb58f5117baa79d50" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584"
I20260927 10:49:02.528354 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c8559ae16f485fb58f5117baa79d50" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b9d51a93e4426dbfb2dadb8d21771d"
I20260927 10:49:02.528357 17197 raft_consensus.cc:3037] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.528496 17057 raft_consensus.cc:3037] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.529492 17057 raft_consensus.cc:2454] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 1.
I20260927 10:49:02.529793 17197 raft_consensus.cc:2454] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 1.
I20260927 10:49:02.529809 17271 leader_election.cc:302] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, 82f87b2e40154883901485d4b8f7d7e0; no voters:
I20260927 10:49:02.529994 17418 raft_consensus.cc:2781] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:02.530251 17418 raft_consensus.cc:686] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 LEADER]: Becoming Leader. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Running, Role: LEADER
I20260927 10:49:02.530579 17418 consensus_queue.cc:237] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.533619 16900 catalog_manager.cc:5495] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131). New cstate: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } health_report { overall_health: HEALTHY } } }
W20260927 10:49:02.544112 17384 tablet.cc:2369] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:49:02.561254 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.557735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.9.39.131:33629)
0927 10:49:02.557882 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.557889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.557910 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:49:02.557993 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.557996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.558047 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.558157 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:02.558393 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.558919 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.559783 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.560003 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.560105 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.560366 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.560786 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.560921 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.561006 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.561056 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 10:49:02.567221 17197 raft_consensus.cc:1261] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Refusing update from remote peer 82f87b2e40154883901485d4b8f7d7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:02.567277 17057 raft_consensus.cc:1261] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Refusing update from remote peer 82f87b2e40154883901485d4b8f7d7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:02.567901 17418 consensus_queue.cc:1035] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:02.568181 17421 consensus_queue.cc:1035] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:02.574906 17424 mvcc.cc:205] Tried to move back new op lower bound from 7333913159947837440 to 7333913159807967232. Current Snapshot: MvccSnapshot[applied={T|T < 7333913159947837440}]
W20260927 10:49:02.575130 17244 tablet.cc:2369] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:49:02.576937 17425 mvcc.cc:205] Tried to move back new op lower bound from 7333913159947837440 to 7333913159807967232. Current Snapshot: MvccSnapshot[applied={T|T < 7333913159947837440}]
I20260927 10:49:02.582722 16898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:02.582890 16898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:02.583817 17426 mvcc.cc:205] Tried to move back new op lower bound from 7333913159947837440 to 7333913159807967232. Current Snapshot: MvccSnapshot[applied={T|T < 7333913159947837440}]
I20260927 10:49:02.585330 16898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Sending DeleteTablet for 3 replicas of tablet b9c8559ae16f485fb58f5117baa79d50
I20260927 10:49:02.586064 17177 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c8559ae16f485fb58f5117baa79d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:02 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40852
I20260927 10:49:02.586217 17037 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c8559ae16f485fb58f5117baa79d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:02 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:46040
I20260927 10:49:02.586467 17317 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c8559ae16f485fb58f5117baa79d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:02 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34656
I20260927 10:49:02.586933 17436 tablet_replica.cc:330] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: stopping tablet replica
I20260927 10:49:02.587133 16898 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 10:49:02.587455 17436 raft_consensus.cc:2229] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 LEADER]: Raft consensus shutting down.
W20260927 10:49:02.587677 16898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:02.587855 17436 raft_consensus.cc:2258] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:02.589131 17436 ts_tablet_manager.cc:1905] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:02.592531 17436 ts_tablet_manager.cc:1918] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:02.592869 17436 log.cc:1196] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/b9c8559ae16f485fb58f5117baa79d50
I20260927 10:49:02.593235 17436 ts_tablet_manager.cc:1939] T b9c8559ae16f485fb58f5117baa79d50 P 82f87b2e40154883901485d4b8f7d7e0: Deleting consensus metadata
I20260927 10:49:02.593547 17037 tablet_service.cc:1467] Processing CreateTablet for tablet 1e065b8dc928474bb2bb8f76c831158a (DEFAULT_TABLE table=test-table [id=1c2eef9e46cc4305a796ada347871f6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:02.593827 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e065b8dc928474bb2bb8f76c831158a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.595216 16885 catalog_manager.cc:4841] TS 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629): tablet b9c8559ae16f485fb58f5117baa79d50 (table test-table [id=8a81c555cacf473cb9bd7f76965ab0c6]) successfully deleted
I20260927 10:49:02.595719 17317 tablet_service.cc:1467] Processing CreateTablet for tablet 1e065b8dc928474bb2bb8f76c831158a (DEFAULT_TABLE table=test-table [id=1c2eef9e46cc4305a796ada347871f6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:02.595959 17317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e065b8dc928474bb2bb8f76c831158a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.596199 17411 tablet_bootstrap.cc:492] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Bootstrap starting.
I20260927 10:49:02.596592 17177 tablet_service.cc:1467] Processing CreateTablet for tablet 1e065b8dc928474bb2bb8f76c831158a (DEFAULT_TABLE table=test-table [id=1c2eef9e46cc4305a796ada347871f6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:02.596858 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e065b8dc928474bb2bb8f76c831158a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.597150 17411 tablet_bootstrap.cc:654] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.598085 17412 tablet_bootstrap.cc:492] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Bootstrap starting.
I20260927 10:49:02.598389 17434 tablet_replica.cc:330] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: stopping tablet replica
I20260927 10:49:02.598719 17434 raft_consensus.cc:2229] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:02.599013 17434 raft_consensus.cc:2258] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:02.599032 17412 tablet_bootstrap.cc:654] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.599470 17414 tablet_bootstrap.cc:492] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Bootstrap starting.
I20260927 10:49:02.600183 17414 tablet_bootstrap.cc:654] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.600847 17435 tablet_replica.cc:330] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: stopping tablet replica
I20260927 10:49:02.601141 17435 raft_consensus.cc:2229] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:02.601432 17435 raft_consensus.cc:2258] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:02.602195 17414 tablet_bootstrap.cc:492] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: No bootstrap required, opened a new log
I20260927 10:49:02.602298 17414 ts_tablet_manager.cc:1397] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 10:49:02.602540 17434 ts_tablet_manager.cc:1905] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:02.602761 17414 raft_consensus.cc:348] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.602900 17414 raft_consensus.cc:374] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.602939 17414 raft_consensus.cc:729] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c14a2977b4a1f890beabf31b3c584, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.603078 17414 consensus_queue.cc:260] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.603335 17414 ts_tablet_manager.cc:1428] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:02.604589 17411 tablet_bootstrap.cc:492] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: No bootstrap required, opened a new log
I20260927 10:49:02.604688 17411 ts_tablet_manager.cc:1397] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260927 10:49:02.605024 17434 ts_tablet_manager.cc:1918] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:02.605103 17434 log.cc:1196] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/b9c8559ae16f485fb58f5117baa79d50
I20260927 10:49:02.605189 17411 raft_consensus.cc:348] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.605295 17411 raft_consensus.cc:374] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.605329 17411 raft_consensus.cc:729] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.605419 17434 ts_tablet_manager.cc:1939] T b9c8559ae16f485fb58f5117baa79d50 P ca6c14a2977b4a1f890beabf31b3c584: Deleting consensus metadata
I20260927 10:49:02.605437 17411 consensus_queue.cc:260] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.605708 17411 ts_tablet_manager.cc:1428] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:02.606606 16885 catalog_manager.cc:4841] TS ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923): tablet b9c8559ae16f485fb58f5117baa79d50 (table test-table [id=8a81c555cacf473cb9bd7f76965ab0c6]) successfully deleted
I20260927 10:49:02.607666 17435 ts_tablet_manager.cc:1905] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:02.610574 17435 ts_tablet_manager.cc:1918] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:02.610675 17435 log.cc:1196] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/b9c8559ae16f485fb58f5117baa79d50
I20260927 10:49:02.611027 17435 ts_tablet_manager.cc:1939] T b9c8559ae16f485fb58f5117baa79d50 P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting consensus metadata
I20260927 10:49:02.611644 17412 tablet_bootstrap.cc:492] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: No bootstrap required, opened a new log
I20260927 10:49:02.611744 17412 ts_tablet_manager.cc:1397] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260927 10:49:02.612305 17412 raft_consensus.cc:348] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.612406 17412 raft_consensus.cc:374] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.612426 17412 raft_consensus.cc:729] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.612370 16887 catalog_manager.cc:4841] TS 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443): tablet b9c8559ae16f485fb58f5117baa79d50 (table test-table [id=8a81c555cacf473cb9bd7f76965ab0c6]) successfully deleted
I20260927 10:49:02.612568 17412 consensus_queue.cc:260] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.612848 17412 ts_tablet_manager.cc:1428] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:02.671625 17417 raft_consensus.cc:482] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:02.671770 17417 raft_consensus.cc:504] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.672959 17417 leader_election.cc:288] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923), 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:02.677553 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.673303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:58671 (local address 127.9.39.130:35923)
0927 10:49:02.673438 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.673443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.673534 (+ 91us) server_negotiation.cc:407] Connection header received
0927 10:49:02.673725 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.673729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.673785 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.673837 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.674282 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.674922 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.676332 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.676949 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.677072 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.677123 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.677356 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.677426 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:49:02.677568 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.673152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35923 (local address 127.9.39.129:58671)
0927 10:49:02.673439 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.673456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.673569 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.673992 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.673997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.674015 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.674137 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.674142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.675040 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.675042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.676191 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.676200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.677090 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.677093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.677222 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.677242 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.677337 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.677414 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":121,"threads_started":1}
I20260927 10:49:02.677698 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.673734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:46473 (local address 127.9.39.131:33629)
0927 10:49:02.673894 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.673898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.673912 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:49:02.674047 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.674050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.674099 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.674200 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:02.674499 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.675303 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.676597 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.676982 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.677114 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.677168 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.677497 (+ 329us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.677575 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260927 10:49:02.678095 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.673577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33629 (local address 127.9.39.129:46473)
0927 10:49:02.673849 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.673862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.673934 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.674241 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.674246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.674268 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:02.674374 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.674388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.675435 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.675439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.676479 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.676486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.677242 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.677246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.677346 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.677370 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.677894 (+ 524us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.677969 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":59,"threads_started":1}
I20260927 10:49:02.678269 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e065b8dc928474bb2bb8f76c831158a" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584" is_pre_election: true
I20260927 10:49:02.678426 17197 raft_consensus.cc:2454] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 0.
I20260927 10:49:02.678620 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e065b8dc928474bb2bb8f76c831158a" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f87b2e40154883901485d4b8f7d7e0" is_pre_election: true
I20260927 10:49:02.678856 17337 raft_consensus.cc:2454] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 0.
I20260927 10:49:02.678867 16989 leader_election.cc:302] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:02.679222 17417 raft_consensus.cc:2781] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:02.679288 17417 raft_consensus.cc:482] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:02.679327 17417 raft_consensus.cc:3037] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.680277 17417 raft_consensus.cc:504] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.680656 17417 leader_election.cc:288] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 election: Requested vote from peers ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923), 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:02.680998 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e065b8dc928474bb2bb8f76c831158a" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584"
I20260927 10:49:02.681034 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e065b8dc928474bb2bb8f76c831158a" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f87b2e40154883901485d4b8f7d7e0"
I20260927 10:49:02.681119 17197 raft_consensus.cc:3037] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.681131 17337 raft_consensus.cc:3037] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.682227 17197 raft_consensus.cc:2454] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 1.
I20260927 10:49:02.682220 17337 raft_consensus.cc:2454] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 1.
I20260927 10:49:02.682497 16989 leader_election.cc:302] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, 82f87b2e40154883901485d4b8f7d7e0; no voters:
I20260927 10:49:02.682683 17417 raft_consensus.cc:2781] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:02.682953 17417 raft_consensus.cc:686] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 LEADER]: Becoming Leader. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Running, Role: LEADER
I20260927 10:49:02.683213 17417 consensus_queue.cc:237] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.685748 16900 catalog_manager.cc:5495] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d reported cstate change: term changed from 0 to 1, leader changed from <none> to 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129). New cstate: current_term: 1 leader_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } health_report { overall_health: UNKNOWN } } }
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.190@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:02.745390 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.736579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.9.39.190:37331)
0927 10:49:02.736765 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.736769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.736938 (+ 169us) server_negotiation.cc:407] Connection header received
0927 10:49:02.737121 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.737123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.737159 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.737243 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.737957 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.738469 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.739364 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.739570 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.739685 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.742148 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:02.742154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:02.742161 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:02.742205 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:02.743652 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.744123 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.744129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.744131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.744208 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:02.744483 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:02.744489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:02.744490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:02.744660 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:02.744763 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.745055 (+ 292us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.745219 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":51}
I20260927 10:49:02.745391 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.736503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:37331 (local address 127.0.0.1:55590)
0927 10:49:02.736847 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.736859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.736999 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.737259 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.737261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.737689 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:02.737828 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.737848 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.738575 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.738581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.739239 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.739247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.739680 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.739683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.739754 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.740402 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:02.740414 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:02.742006 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:02.743773 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.743784 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.743790 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.744029 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.744298 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:02.744301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:02.744302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:02.744399 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:02.744763 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:02.744771 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:02.744884 (+ 113us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.745005 (+ 121us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.745158 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":146,"threads_started":1}
I20260927 10:49:02.750013 16900 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:55590:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 10:49:02.750650 16900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:02.755648 17037 tablet_service.cc:1467] Processing CreateTablet for tablet eb29e7acc5894d5c9da22cdf0655de05 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f379b7f92d3f4a34921917b11a0c8613]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:49:02.755714 17317 tablet_service.cc:1467] Processing CreateTablet for tablet eb29e7acc5894d5c9da22cdf0655de05 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f379b7f92d3f4a34921917b11a0c8613]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:49:02.755734 17177 tablet_service.cc:1467] Processing CreateTablet for tablet eb29e7acc5894d5c9da22cdf0655de05 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f379b7f92d3f4a34921917b11a0c8613]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 10:49:02.756053 17317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb29e7acc5894d5c9da22cdf0655de05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.756053 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb29e7acc5894d5c9da22cdf0655de05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.756116 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb29e7acc5894d5c9da22cdf0655de05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:02.758836 17412 tablet_bootstrap.cc:492] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: Bootstrap starting.
I20260927 10:49:02.758900 17414 tablet_bootstrap.cc:492] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: Bootstrap starting.
I20260927 10:49:02.759604 17412 tablet_bootstrap.cc:654] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.759781 17414 tablet_bootstrap.cc:654] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.759820 17411 tablet_bootstrap.cc:492] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: Bootstrap starting.
I20260927 10:49:02.760717 17411 tablet_bootstrap.cc:654] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:02.760814 17412 tablet_bootstrap.cc:492] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: No bootstrap required, opened a new log
I20260927 10:49:02.760900 17412 ts_tablet_manager.cc:1397] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:02.761293 17412 raft_consensus.cc:348] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.761432 17412 raft_consensus.cc:374] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.761471 17412 raft_consensus.cc:729] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.761612 17412 consensus_queue.cc:260] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.761936 17412 ts_tablet_manager.cc:1428] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:02.761879 17418 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.762013 17418 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.762161 17414 tablet_bootstrap.cc:492] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: No bootstrap required, opened a new log
I20260927 10:49:02.762171 17411 tablet_bootstrap.cc:492] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: No bootstrap required, opened a new log
I20260927 10:49:02.762238 17411 ts_tablet_manager.cc:1397] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260927 10:49:02.762238 17414 ts_tablet_manager.cc:1397] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:49:02.762645 17411 raft_consensus.cc:348] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.762656 17414 raft_consensus.cc:348] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.762782 17414 raft_consensus.cc:374] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.762782 17411 raft_consensus.cc:374] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:02.762817 17411 raft_consensus.cc:729] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.762817 17414 raft_consensus.cc:729] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c14a2977b4a1f890beabf31b3c584, State: Initialized, Role: FOLLOWER
I20260927 10:49:02.762943 17411 consensus_queue.cc:260] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.762960 17414 consensus_queue.cc:260] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.763213 17411 ts_tablet_manager.cc:1428] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:02.763200 17419 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.763351 17419 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.763345 17417 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.763440 17417 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.763200 17414 ts_tablet_manager.cc:1428] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260927 10:49:02.853904 17104 tablet.cc:2369] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:49:02.860597 17418 raft_consensus.cc:482] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:02.860754 17418 raft_consensus.cc:504] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.861173 17418 leader_election.cc:288] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443), ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923)
I20260927 10:49:02.861618 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29e7acc5894d5c9da22cdf0655de05" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" is_pre_election: true
I20260927 10:49:02.861639 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29e7acc5894d5c9da22cdf0655de05" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584" is_pre_election: true
I20260927 10:49:02.861773 17057 raft_consensus.cc:2454] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 0.
I20260927 10:49:02.861791 17197 raft_consensus.cc:2454] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 0.
I20260927 10:49:02.862061 17271 leader_election.cc:302] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, 82f87b2e40154883901485d4b8f7d7e0; no voters:
I20260927 10:49:02.862249 17418 raft_consensus.cc:2781] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:02.862321 17418 raft_consensus.cc:482] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:02.862350 17418 raft_consensus.cc:3037] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.863464 17418 raft_consensus.cc:504] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.863797 17418 leader_election.cc:288] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 election: Requested vote from peers 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443), ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923)
I20260927 10:49:02.864099 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29e7acc5894d5c9da22cdf0655de05" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584"
I20260927 10:49:02.864125 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29e7acc5894d5c9da22cdf0655de05" candidate_uuid: "82f87b2e40154883901485d4b8f7d7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b9d51a93e4426dbfb2dadb8d21771d"
I20260927 10:49:02.864240 17057 raft_consensus.cc:3037] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.864241 17197 raft_consensus.cc:3037] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:02.865116 17197 raft_consensus.cc:2454] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 1.
I20260927 10:49:02.865401 17057 raft_consensus.cc:2454] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f87b2e40154883901485d4b8f7d7e0 in term 1.
I20260927 10:49:02.865401 17269 leader_election.cc:302] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82f87b2e40154883901485d4b8f7d7e0, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:02.865579 17418 raft_consensus.cc:2781] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:02.865653 17418 raft_consensus.cc:686] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 LEADER]: Becoming Leader. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Running, Role: LEADER
I20260927 10:49:02.865794 17418 consensus_queue.cc:237] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:02.866484 17418 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: TxnStatusTablet state changed. Reason: New leader 82f87b2e40154883901485d4b8f7d7e0. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.866580 17418 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:49:02.867054 17454 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 10:49:02.867205 17454 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 10:49:02.867622 16899 catalog_manager.cc:5495] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131). New cstate: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } health_report { overall_health: HEALTHY } } }
I20260927 10:49:02.883761 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.880377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33629 (local address 127.0.0.1:34676)
0927 10:49:02.880517 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.880533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.880634 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.880906 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.880908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.880915 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:49:02.881000 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.881004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.881965 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.881968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.882565 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.882572 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.883118 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.883121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.883209 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.883251 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.883534 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.883543 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.883592 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.883655 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 10:49:02.883785 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.880474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.9.39.131:33629)
0927 10:49:02.880580 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.880585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.880598 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:02.880746 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.880749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.880802 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.880886 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:02.881092 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.881862 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.882663 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.883021 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.883121 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.883368 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.883445 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.883521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.883601 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.883650 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 10:49:02.886155 17197 raft_consensus.cc:1261] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Refusing update from remote peer 82f87b2e40154883901485d4b8f7d7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:02.886157 17057 raft_consensus.cc:1261] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Refusing update from remote peer 82f87b2e40154883901485d4b8f7d7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:02.886576 17421 consensus_queue.cc:1035] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:02.886754 17418 consensus_queue.cc:1035] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:02.887991 17417 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: TxnStatusTablet state changed. Reason: New leader 82f87b2e40154883901485d4b8f7d7e0. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.888105 17417 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.888247 17419 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: TxnStatusTablet state changed. Reason: New leader 82f87b2e40154883901485d4b8f7d7e0. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.888356 17419 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.889652 17418 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.889760 17418 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:49:02.890187 17419 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.890278 17419 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P ca6c14a2977b4a1f890beabf31b3c584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.893297 17418 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.893405 17418 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 82f87b2e40154883901485d4b8f7d7e0: This TxnStatusTablet replica's current role is: LEADER
I20260927 10:49:02.894888 17417 tablet_replica.cc:439] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82f87b2e40154883901485d4b8f7d7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } } }
I20260927 10:49:02.894999 17417 tablet_replica.cc:442] T eb29e7acc5894d5c9da22cdf0655de05 P 71b9d51a93e4426dbfb2dadb8d21771d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 10:49:02.898649 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.894790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.9.39.129:37443)
0927 10:49:02.895128 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.895132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.895145 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:02.895216 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.895219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.895276 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.895381 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:02.895615 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.896300 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.897120 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.897436 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.897533 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.897793 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.898276 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.898402 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.898468 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.898520 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260927 10:49:02.909081 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.905449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:58641 (local address 127.9.39.131:33629)
0927 10:49:02.905551 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.905556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.905741 (+ 185us) server_negotiation.cc:407] Connection header received
0927 10:49:02.905820 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.905823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.905885 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.905968 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:02.906180 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.906968 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.907951 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.908285 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.908391 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.908673 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.908748 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.908818 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.908904 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.908956 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 10:49:02.909082 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.905388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33629 (local address 127.9.39.129:58641)
0927 10:49:02.905658 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.905671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.905753 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.905980 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.905982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.905988 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:49:02.906076 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.906080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.907072 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.907074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.907841 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.907850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.908403 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.908406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.908475 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.908513 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.908831 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.908836 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.908889 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.908947 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":101,"threads_started":1}
I20260927 10:49:02.916255 17337 raft_consensus.cc:1261] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Refusing update from remote peer 71b9d51a93e4426dbfb2dadb8d21771d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:02.916729 17197 raft_consensus.cc:1261] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Refusing update from remote peer 71b9d51a93e4426dbfb2dadb8d21771d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:02.916802 17417 consensus_queue.cc:1035] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:02.917120 17442 consensus_queue.cc:1035] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:02.939311 17467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.935123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37443 (local address 127.9.39.131:34949)
0927 10:49:02.935515 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:02.935531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:02.935669 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:02.936033 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:02.936038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:02.936048 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 10:49:02.936197 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.936206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.937236 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.937238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.938009 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:02.938019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.938559 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.938562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:02.938624 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.938669 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.939026 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.939035 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 10:49:02.939099 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 10:49:02.939177 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":100,"threads_started":1}
I20260927 10:49:02.939306 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:02.935360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34949 (local address 127.9.39.129:37443)
0927 10:49:02.935481 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.935485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.935593 (+ 108us) server_negotiation.cc:407] Connection header received
0927 10:49:02.935789 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.935792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.935844 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.935890 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:02.936324 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.937126 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.938108 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.938449 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.938565 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.938834 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:02.938917 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:02.938986 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:02.939110 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 10:49:02.939169 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:49:02.961560 16900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:02.961797 16900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:02.963960 16900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Sending DeleteTablet for 3 replicas of tablet 1e065b8dc928474bb2bb8f76c831158a
I20260927 10:49:02.964461 17037 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e065b8dc928474bb2bb8f76c831158a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:02 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:46040
I20260927 10:49:02.964653 17435 tablet_replica.cc:330] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: stopping tablet replica
I20260927 10:49:02.964725 17177 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e065b8dc928474bb2bb8f76c831158a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:02 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40852
I20260927 10:49:02.964766 17435 raft_consensus.cc:2229] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:49:02.964882 17434 tablet_replica.cc:330] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: stopping tablet replica
I20260927 10:49:02.964967 17434 raft_consensus.cc:2229] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:02.965054 17434 raft_consensus.cc:2258] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:02.965056 17435 raft_consensus.cc:2258] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:02.965425 17317 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e065b8dc928474bb2bb8f76c831158a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:02 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34656
I20260927 10:49:02.965593 17436 tablet_replica.cc:330] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: stopping tablet replica
I20260927 10:49:02.965715 17436 raft_consensus.cc:2229] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:02.965826 17436 raft_consensus.cc:2258] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:02.966645 17434 ts_tablet_manager.cc:1905] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:02.969820 17434 ts_tablet_manager.cc:1918] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:49:02.969903 17434 log.cc:1196] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/1e065b8dc928474bb2bb8f76c831158a
I20260927 10:49:02.970196 17434 ts_tablet_manager.cc:1939] T 1e065b8dc928474bb2bb8f76c831158a P ca6c14a2977b4a1f890beabf31b3c584: Deleting consensus metadata
I20260927 10:49:02.971169 16885 catalog_manager.cc:4841] TS ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923): tablet 1e065b8dc928474bb2bb8f76c831158a (table test-table [id=1c2eef9e46cc4305a796ada347871f6c]) successfully deleted
I20260927 10:49:02.971489 17435 ts_tablet_manager.cc:1905] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:02.973568 17435 ts_tablet_manager.cc:1918] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:49:02.973676 17435 log.cc:1196] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/1e065b8dc928474bb2bb8f76c831158a
I20260927 10:49:02.973910 17435 ts_tablet_manager.cc:1939] T 1e065b8dc928474bb2bb8f76c831158a P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting consensus metadata
I20260927 10:49:02.974534 16887 catalog_manager.cc:4841] TS 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443): tablet 1e065b8dc928474bb2bb8f76c831158a (table test-table [id=1c2eef9e46cc4305a796ada347871f6c]) successfully deleted
I20260927 10:49:02.975232 17436 ts_tablet_manager.cc:1905] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:02.978143 17436 ts_tablet_manager.cc:1918] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:49:02.978226 17436 log.cc:1196] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/1e065b8dc928474bb2bb8f76c831158a
I20260927 10:49:02.978471 17436 ts_tablet_manager.cc:1939] T 1e065b8dc928474bb2bb8f76c831158a P 82f87b2e40154883901485d4b8f7d7e0: Deleting consensus metadata
I20260927 10:49:02.979204 16885 catalog_manager.cc:4841] TS 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629): tablet 1e065b8dc928474bb2bb8f76c831158a (table test-table [id=1c2eef9e46cc4305a796ada347871f6c]) successfully deleted
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:02 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 10:49:02.999043 16949 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:02.992476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55598 (local address 127.9.39.190:37331)
0927 10:49:02.992607 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:02.992612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:02.992863 (+ 251us) server_negotiation.cc:407] Connection header received
0927 10:49:02.993028 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:02.993031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:02.993085 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:02.993178 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:02.993989 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.994519 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.995455 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:02.995691 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:02.995804 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:02.998919 (+ 3115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55598: BlockingRecv error: recv got EOF from 127.0.0.1:55598 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 10:49:03 dist-test-slave-p9q6 krb5kdc[16855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM
I20260927 10:49:03.011512 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.000672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55600 (local address 127.9.39.190:37331)
0927 10:49:03.000786 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:03.000790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:03.002971 (+ 2181us) server_negotiation.cc:407] Connection header received
0927 10:49:03.003096 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:03.003099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:03.003162 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:03.003250 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:03.003915 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.004551 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.005329 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.005556 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.005663 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.008305 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:03.008309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:03.008312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:03.008346 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:03.009795 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:03.010230 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:03.010234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:03.010236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:03.010316 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:03.010593 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:03.010596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:03.010598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:03.010855 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:03.010991 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:03.011230 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 10:49:03.011341 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:49:03.014155 16900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:49:03.014650 16900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:03.020537 17037 tablet_service.cc:1467] Processing CreateTablet for tablet 004371751baf4b33b3aaa4bce090023c (DEFAULT_TABLE table=test-table [id=819235ac2ed043f2b9c18013eb4da41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:03.020871 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 004371751baf4b33b3aaa4bce090023c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:03.020994 17177 tablet_service.cc:1467] Processing CreateTablet for tablet 004371751baf4b33b3aaa4bce090023c (DEFAULT_TABLE table=test-table [id=819235ac2ed043f2b9c18013eb4da41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:03.021266 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 004371751baf4b33b3aaa4bce090023c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:03.023602 17411 tablet_bootstrap.cc:492] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Bootstrap starting.
I20260927 10:49:03.023854 17317 tablet_service.cc:1467] Processing CreateTablet for tablet 004371751baf4b33b3aaa4bce090023c (DEFAULT_TABLE table=test-table [id=819235ac2ed043f2b9c18013eb4da41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:03.024114 17317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 004371751baf4b33b3aaa4bce090023c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:03.024560 17411 tablet_bootstrap.cc:654] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:03.026805 17412 tablet_bootstrap.cc:492] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Bootstrap starting.
I20260927 10:49:03.026947 17411 tablet_bootstrap.cc:492] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: No bootstrap required, opened a new log
I20260927 10:49:03.027017 17411 ts_tablet_manager.cc:1397] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:49:03.027480 17411 raft_consensus.cc:348] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.027604 17411 raft_consensus.cc:374] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:03.027657 17411 raft_consensus.cc:729] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Initialized, Role: FOLLOWER
I20260927 10:49:03.027666 17412 tablet_bootstrap.cc:654] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:03.027809 17411 consensus_queue.cc:260] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.028080 17411 ts_tablet_manager.cc:1428] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:03.028728 17414 tablet_bootstrap.cc:492] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Bootstrap starting.
I20260927 10:49:03.029718 17414 tablet_bootstrap.cc:654] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:03.031527 17412 tablet_bootstrap.cc:492] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: No bootstrap required, opened a new log
I20260927 10:49:03.031616 17412 ts_tablet_manager.cc:1397] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260927 10:49:03.032095 17412 raft_consensus.cc:348] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.032227 17412 raft_consensus.cc:374] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:03.032262 17412 raft_consensus.cc:729] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Initialized, Role: FOLLOWER
I20260927 10:49:03.032394 17412 consensus_queue.cc:260] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.032665 17412 ts_tablet_manager.cc:1428] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 10:49:03.039654 17414 tablet_bootstrap.cc:492] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: No bootstrap required, opened a new log
I20260927 10:49:03.039776 17414 ts_tablet_manager.cc:1397] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260927 10:49:03.040218 17414 raft_consensus.cc:348] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.040352 17414 raft_consensus.cc:374] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:03.040395 17414 raft_consensus.cc:729] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c14a2977b4a1f890beabf31b3c584, State: Initialized, Role: FOLLOWER
I20260927 10:49:03.040524 17414 consensus_queue.cc:260] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.040789 17414 ts_tablet_manager.cc:1428] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:03.080507 17419 raft_consensus.cc:482] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:03.080646 17419 raft_consensus.cc:504] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.081761 17419 leader_election.cc:288] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443), 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:03.087216 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.082495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:57331 (local address 127.9.39.129:37443)
0927 10:49:03.082618 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:03.082622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:03.082895 (+ 273us) server_negotiation.cc:407] Connection header received
0927 10:49:03.083023 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:03.083026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:03.083089 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:03.083188 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:03.083571 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.084358 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.085949 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.086537 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.086620 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.086671 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:03.087023 (+ 352us) server_negotiation.cc:299] Negotiation successful
0927 10:49:03.087092 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:49:03.087628 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.082021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37443 (local address 127.9.39.130:57331)
0927 10:49:03.082779 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:03.082793 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:03.082921 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:03.083307 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:03.083309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:03.083328 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:03.083440 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:03.083445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.084511 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.084514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:03.085785 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:03.085795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.086786 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.086789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:03.086896 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.086918 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:49:03.087419 (+ 501us) client_negotiation.cc:240] Negotiation successful
0927 10:49:03.087491 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":73,"threads_started":1}
I20260927 10:49:03.087759 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.082404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:49407 (local address 127.9.39.131:33629)
0927 10:49:03.082519 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:03.082524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:03.083222 (+ 698us) server_negotiation.cc:407] Connection header received
0927 10:49:03.083405 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:03.083408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:03.083453 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:03.083501 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:03.083910 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.084651 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.086165 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.086804 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.086900 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.086951 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:03.087534 (+ 583us) server_negotiation.cc:299] Negotiation successful
0927 10:49:03.087623 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 10:49:03.088281 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "004371751baf4b33b3aaa4bce090023c" candidate_uuid: "ca6c14a2977b4a1f890beabf31b3c584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" is_pre_election: true
I20260927 10:49:03.088454 17057 raft_consensus.cc:2454] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c14a2977b4a1f890beabf31b3c584 in term 0.
I20260927 10:49:03.088523 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.082067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:33629 (local address 127.9.39.130:49407)
0927 10:49:03.083128 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:03.083141 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:03.083288 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:03.083638 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:03.083641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:03.083655 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:03.083770 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:03.083776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.084797 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.084800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:03.086024 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:03.086035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.087046 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.087049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:03.087147 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.087170 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 10:49:03.088318 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0927 10:49:03.088391 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260927 10:49:03.088840 17131 leader_election.cc:302] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:03.089088 17419 raft_consensus.cc:2781] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:03.089146 17419 raft_consensus.cc:482] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:03.089179 17419 raft_consensus.cc:3037] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:03.089304 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "004371751baf4b33b3aaa4bce090023c" candidate_uuid: "ca6c14a2977b4a1f890beabf31b3c584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f87b2e40154883901485d4b8f7d7e0" is_pre_election: true
I20260927 10:49:03.089438 17337 raft_consensus.cc:2454] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c14a2977b4a1f890beabf31b3c584 in term 0.
I20260927 10:49:03.090324 17419 raft_consensus.cc:504] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.090653 17419 leader_election.cc:288] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [CANDIDATE]: Term 1 election: Requested vote from peers 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443), 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:03.091043 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "004371751baf4b33b3aaa4bce090023c" candidate_uuid: "ca6c14a2977b4a1f890beabf31b3c584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b9d51a93e4426dbfb2dadb8d21771d"
I20260927 10:49:03.091153 17057 raft_consensus.cc:3037] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:03.091666 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "004371751baf4b33b3aaa4bce090023c" candidate_uuid: "ca6c14a2977b4a1f890beabf31b3c584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f87b2e40154883901485d4b8f7d7e0"
I20260927 10:49:03.091806 17337 raft_consensus.cc:3037] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:03.092322 17057 raft_consensus.cc:2454] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c14a2977b4a1f890beabf31b3c584 in term 1.
I20260927 10:49:03.093022 17337 raft_consensus.cc:2454] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c14a2977b4a1f890beabf31b3c584 in term 1.
I20260927 10:49:03.092592 17131 leader_election.cc:302] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:03.093513 17419 raft_consensus.cc:2781] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:03.093724 17419 raft_consensus.cc:686] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 1 LEADER]: Becoming Leader. State: Replica: ca6c14a2977b4a1f890beabf31b3c584, State: Running, Role: LEADER
I20260927 10:49:03.093995 17419 consensus_queue.cc:237] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.096458 16899 catalog_manager.cc:5495] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130). New cstate: current_term: 1 leader_uuid: "ca6c14a2977b4a1f890beabf31b3c584" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } health_report { overall_health: UNKNOWN } } }
I20260927 10:49:03.114082 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.110477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.9.39.130:35923)
0927 10:49:03.110606 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:03.110611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:03.110624 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:03.110768 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:03.110771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:03.110839 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:03.110942 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:03.111187 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.111743 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.112626 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.112896 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.113001 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.113267 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:03.113697 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:03.113829 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:03.113889 (+ 60us) server_negotiation.cc:299] Negotiation successful
0927 10:49:03.113938 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:49:03.116684 17057 raft_consensus.cc:1261] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Refusing update from remote peer ca6c14a2977b4a1f890beabf31b3c584: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:03.117040 17337 raft_consensus.cc:1261] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Refusing update from remote peer ca6c14a2977b4a1f890beabf31b3c584: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:03.117271 17419 consensus_queue.cc:1035] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:03.117507 17487 consensus_queue.cc:1035] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:03.128957 16899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:03.129104 16899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:03.131151 16899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Sending DeleteTablet for 3 replicas of tablet 004371751baf4b33b3aaa4bce090023c
I20260927 10:49:03.131678 17037 tablet_service.cc:1514] Processing DeleteTablet for tablet 004371751baf4b33b3aaa4bce090023c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:03 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:46040
I20260927 10:49:03.131815 17317 tablet_service.cc:1514] Processing DeleteTablet for tablet 004371751baf4b33b3aaa4bce090023c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:03 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34656
I20260927 10:49:03.131870 17435 tablet_replica.cc:330] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: stopping tablet replica
I20260927 10:49:03.131975 17435 raft_consensus.cc:2229] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:03.132102 17435 raft_consensus.cc:2258] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:03.132268 17436 tablet_replica.cc:330] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: stopping tablet replica
I20260927 10:49:03.132387 17436 raft_consensus.cc:2229] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:03.132484 17436 raft_consensus.cc:2258] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:03.133069 17177 tablet_service.cc:1514] Processing DeleteTablet for tablet 004371751baf4b33b3aaa4bce090023c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:03 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40852
I20260927 10:49:03.133244 17434 tablet_replica.cc:330] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: stopping tablet replica
I20260927 10:49:03.133348 17434 raft_consensus.cc:2229] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:49:03.133416 17436 ts_tablet_manager.cc:1905] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:03.133630 17434 raft_consensus.cc:2258] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:03.134275 17434 ts_tablet_manager.cc:1905] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:03.136781 17434 ts_tablet_manager.cc:1918] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:03.136857 17434 log.cc:1196] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/004371751baf4b33b3aaa4bce090023c
I20260927 10:49:03.137111 17434 ts_tablet_manager.cc:1939] T 004371751baf4b33b3aaa4bce090023c P ca6c14a2977b4a1f890beabf31b3c584: Deleting consensus metadata
I20260927 10:49:03.137466 17436 ts_tablet_manager.cc:1918] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:03.137535 17436 log.cc:1196] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/004371751baf4b33b3aaa4bce090023c
I20260927 10:49:03.137779 17436 ts_tablet_manager.cc:1939] T 004371751baf4b33b3aaa4bce090023c P 82f87b2e40154883901485d4b8f7d7e0: Deleting consensus metadata
I20260927 10:49:03.138404 16885 catalog_manager.cc:4841] TS 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629): tablet 004371751baf4b33b3aaa4bce090023c (table test-table [id=819235ac2ed043f2b9c18013eb4da41e]) successfully deleted
I20260927 10:49:03.138592 16885 catalog_manager.cc:4841] TS ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923): tablet 004371751baf4b33b3aaa4bce090023c (table test-table [id=819235ac2ed043f2b9c18013eb4da41e]) successfully deleted
I20260927 10:49:03.138839 16899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:49:03.139281 16899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:03.140430 17435 ts_tablet_manager.cc:1905] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:03.142887 17435 ts_tablet_manager.cc:1918] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:03.142968 17435 log.cc:1196] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/004371751baf4b33b3aaa4bce090023c
I20260927 10:49:03.143225 17435 ts_tablet_manager.cc:1939] T 004371751baf4b33b3aaa4bce090023c P 71b9d51a93e4426dbfb2dadb8d21771d: Deleting consensus metadata
I20260927 10:49:03.143997 16887 catalog_manager.cc:4841] TS 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443): tablet 004371751baf4b33b3aaa4bce090023c (table test-table [id=819235ac2ed043f2b9c18013eb4da41e]) successfully deleted
I20260927 10:49:03.144481 17037 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7af030c7f84aa8975147e36895e440 (DEFAULT_TABLE table=test-table [id=a0ed73ff0b5943cea9991f9933a740cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:03.144655 17317 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7af030c7f84aa8975147e36895e440 (DEFAULT_TABLE table=test-table [id=a0ed73ff0b5943cea9991f9933a740cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:03.144723 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7af030c7f84aa8975147e36895e440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:03.144886 17317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7af030c7f84aa8975147e36895e440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:03.146977 17411 tablet_bootstrap.cc:492] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d: Bootstrap starting.
I20260927 10:49:03.147861 17411 tablet_bootstrap.cc:654] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:03.148058 17177 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7af030c7f84aa8975147e36895e440 (DEFAULT_TABLE table=test-table [id=a0ed73ff0b5943cea9991f9933a740cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:03.148315 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7af030c7f84aa8975147e36895e440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:03.148479 17412 tablet_bootstrap.cc:492] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Bootstrap starting.
I20260927 10:49:03.149255 17412 tablet_bootstrap.cc:654] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:03.150763 17411 tablet_bootstrap.cc:492] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d: No bootstrap required, opened a new log
I20260927 10:49:03.150770 17414 tablet_bootstrap.cc:492] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Bootstrap starting.
I20260927 10:49:03.150853 17411 ts_tablet_manager.cc:1397] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 10:49:03.151319 17411 raft_consensus.cc:348] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.151425 17411 raft_consensus.cc:374] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:03.151458 17411 raft_consensus.cc:729] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Initialized, Role: FOLLOWER
I20260927 10:49:03.151584 17411 consensus_queue.cc:260] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.151674 17414 tablet_bootstrap.cc:654] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:03.151894 17411 ts_tablet_manager.cc:1428] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:03.154052 17414 tablet_bootstrap.cc:492] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: No bootstrap required, opened a new log
I20260927 10:49:03.154160 17414 ts_tablet_manager.cc:1397] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 10:49:03.154536 17414 raft_consensus.cc:348] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.154649 17414 raft_consensus.cc:374] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:03.154688 17414 raft_consensus.cc:729] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c14a2977b4a1f890beabf31b3c584, State: Initialized, Role: FOLLOWER
I20260927 10:49:03.154861 17414 consensus_queue.cc:260] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.155123 17414 ts_tablet_manager.cc:1428] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:03.156565 17412 tablet_bootstrap.cc:492] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: No bootstrap required, opened a new log
I20260927 10:49:03.156656 17412 ts_tablet_manager.cc:1397] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260927 10:49:03.157138 17412 raft_consensus.cc:348] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.157253 17412 raft_consensus.cc:374] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:03.157289 17412 raft_consensus.cc:729] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f87b2e40154883901485d4b8f7d7e0, State: Initialized, Role: FOLLOWER
I20260927 10:49:03.157433 17412 consensus_queue.cc:260] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.157687 17412 ts_tablet_manager.cc:1428] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:49:03.482216 17464 raft_consensus.cc:482] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:03.482381 17464 raft_consensus.cc:504] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.482851 17464 leader_election.cc:288] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923), 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:03.483433 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7af030c7f84aa8975147e36895e440" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f87b2e40154883901485d4b8f7d7e0" is_pre_election: true
I20260927 10:49:03.483433 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7af030c7f84aa8975147e36895e440" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584" is_pre_election: true
I20260927 10:49:03.483569 17197 raft_consensus.cc:2454] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 0.
I20260927 10:49:03.483598 17337 raft_consensus.cc:2454] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 0.
I20260927 10:49:03.483922 16989 leader_election.cc:302] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:03.484133 17464 raft_consensus.cc:2781] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:03.484205 17464 raft_consensus.cc:482] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:03.484261 17464 raft_consensus.cc:3037] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:03.485299 17464 raft_consensus.cc:504] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.485630 17464 leader_election.cc:288] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 election: Requested vote from peers ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923), 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629)
I20260927 10:49:03.485997 17197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7af030c7f84aa8975147e36895e440" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c14a2977b4a1f890beabf31b3c584"
I20260927 10:49:03.486101 17197 raft_consensus.cc:3037] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:03.486064 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7af030c7f84aa8975147e36895e440" candidate_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f87b2e40154883901485d4b8f7d7e0"
I20260927 10:49:03.486208 17337 raft_consensus.cc:3037] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:03.487071 17197 raft_consensus.cc:2454] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 1.
I20260927 10:49:03.487252 17337 raft_consensus.cc:2454] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b9d51a93e4426dbfb2dadb8d21771d in term 1.
I20260927 10:49:03.487433 16989 leader_election.cc:302] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b9d51a93e4426dbfb2dadb8d21771d, ca6c14a2977b4a1f890beabf31b3c584; no voters:
I20260927 10:49:03.487634 17464 raft_consensus.cc:2781] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:03.487847 17464 raft_consensus.cc:686] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 LEADER]: Becoming Leader. State: Replica: 71b9d51a93e4426dbfb2dadb8d21771d, State: Running, Role: LEADER
I20260927 10:49:03.488008 17464 consensus_queue.cc:237] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } }
I20260927 10:49:03.489670 16899 catalog_manager.cc:5495] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d reported cstate change: term changed from 0 to 1, leader changed from <none> to 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129). New cstate: current_term: 1 leader_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9d51a93e4426dbfb2dadb8d21771d" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 } health_report { overall_health: UNKNOWN } } }
I20260927 10:49:03.531746 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.528885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.9.39.129:37443)
0927 10:49:03.529007 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:03.529011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:03.529023 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:49:03.529060 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:03.529062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:03.529099 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:03.529165 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:03.529473 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.529966 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.530797 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:03.531010 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:03.531093 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:03.531323 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:03.531385 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:03.531438 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:03.531602 (+ 164us) server_negotiation.cc:299] Negotiation successful
0927 10:49:03.531646 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:49:03.537447 17197 raft_consensus.cc:1261] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Refusing update from remote peer 71b9d51a93e4426dbfb2dadb8d21771d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:03.537451 17337 raft_consensus.cc:1261] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Refusing update from remote peer 71b9d51a93e4426dbfb2dadb8d21771d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:03.537853 17496 consensus_queue.cc:1035] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c14a2977b4a1f890beabf31b3c584" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:03.538079 17464 consensus_queue.cc:1035] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82f87b2e40154883901485d4b8f7d7e0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:03.571053 16899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:03.571226 16899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:03.573314 16899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43bb8a518fe4483bab5a27063e87c1a3: Sending DeleteTablet for 3 replicas of tablet 7f7af030c7f84aa8975147e36895e440
I20260927 10:49:03.573809 17037 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7af030c7f84aa8975147e36895e440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:03 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:46040
I20260927 10:49:03.573982 17435 tablet_replica.cc:330] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d: stopping tablet replica
I20260927 10:49:03.573983 17177 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7af030c7f84aa8975147e36895e440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:03 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:40852
I20260927 10:49:03.574092 17435 raft_consensus.cc:2229] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:49:03.574170 17434 tablet_replica.cc:330] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: stopping tablet replica
I20260927 10:49:03.574266 17435 raft_consensus.cc:2258] T 7f7af030c7f84aa8975147e36895e440 P 71b9d51a93e4426dbfb2dadb8d21771d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:03.574321 17434 raft_consensus.cc:2229] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:03.574447 17434 raft_consensus.cc:2258] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:03.574769 17317 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7af030c7f84aa8975147e36895e440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:03 UTC) from {username='oryx', principal='oryx/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34656
I20260927 10:49:03.574930 17436 tablet_replica.cc:330] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: stopping tablet replica
I20260927 10:49:03.575037 17436 raft_consensus.cc:2229] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:03.575098 17434 ts_tablet_manager.cc:1905] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:03.575150 17436 raft_consensus.cc:2258] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:03.575593 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 16972
I20260927 10:49:03.577462 17434 ts_tablet_manager.cc:1918] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:49:03.577539 17434 log.cc:1196] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/7f7af030c7f84aa8975147e36895e440
I20260927 10:49:03.577773 17434 ts_tablet_manager.cc:1939] T 7f7af030c7f84aa8975147e36895e440 P ca6c14a2977b4a1f890beabf31b3c584: Deleting consensus metadata
I20260927 10:49:03.578532 16885 catalog_manager.cc:4841] TS ca6c14a2977b4a1f890beabf31b3c584 (127.9.39.130:35923): tablet 7f7af030c7f84aa8975147e36895e440 (table test-table [id=a0ed73ff0b5943cea9991f9933a740cb]) successfully deleted
I20260927 10:49:03.579520 17436 ts_tablet_manager.cc:1905] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:03.581735 17436 ts_tablet_manager.cc:1918] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 10:49:03.581810 17436 log.cc:1196] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/7f7af030c7f84aa8975147e36895e440
I20260927 10:49:03.582036 17436 ts_tablet_manager.cc:1939] T 7f7af030c7f84aa8975147e36895e440 P 82f87b2e40154883901485d4b8f7d7e0: Deleting consensus metadata
I20260927 10:49:03.582654 16885 catalog_manager.cc:4841] TS 82f87b2e40154883901485d4b8f7d7e0 (127.9.39.131:33629): tablet 7f7af030c7f84aa8975147e36895e440 (table test-table [id=a0ed73ff0b5943cea9991f9933a740cb]) successfully deleted
W20260927 10:49:03.587909 16887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:37443 (error 108)
I20260927 10:49:03.588061 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 17112
I20260927 10:49:03.589517 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:03.588936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37443 (local address 127.0.0.1:46062)
0927 10:49:03.589388 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:03.589445 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":55,"threads_started":1}
W20260927 10:49:03.589643 16887 catalog_manager.cc:4584] TS 71b9d51a93e4426dbfb2dadb8d21771d (127.9.39.129:37443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f7af030c7f84aa8975147e36895e440: Network error: Client connection negotiation failed: client connection to 127.9.39.129:37443: connect: Connection refused (error 111)
I20260927 10:49:03.595037 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 17252
I20260927 10:49:03.601682 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 16868
2026-09-27T10:49:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3411 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:49:03 dist-test-slave-p9q6 krb5kdc[17509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:49:03 dist-test-slave-p9q6 krb5kdc[17509](info): set up 2 sockets
Sep 27 10:49:03 dist-test-slave-p9q6 krb5kdc[17509](info): commencing operation
krb5kdc: starting...
W20260927 10:49:04.647447 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:49:04 dist-test-slave-p9q6 krb5kdc[17509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:49:04.772203 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43645
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:38627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:43645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:04.880895 17522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:04.881212 17522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:04.881299 17522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:04.885360 17522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:04.885447 17522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:04.885486 17522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:04.885546 17522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:04.885586 17522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:04.891161 17522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:38627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:43645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:04.892310 17522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:04.893361 17522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:04.901780 17530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:04.901789 17528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:04.901789 17527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:04.902036 17522 server_base.cc:1034] running on GCE node
I20260927 10:49:04.902755 17522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:04.903872 17522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:04.905066 17522 hybrid_clock.cc:648] HybridClock initialized: now 1790506144905032 us; error 52 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T10:49:04Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1307 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:49:04 dist-test-slave-p9q6 krb5kdc[17535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:49:04 dist-test-slave-p9q6 krb5kdc[17535](info): set up 2 sockets
Sep 27 10:49:04 dist-test-slave-p9q6 krb5kdc[17535](info): commencing operation
krb5kdc: starting...
W20260927 10:49:05.951655 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:49:06.078192 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43935
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:06.185993 17548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:06.186265 17548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:06.186305 17548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:06.190212 17548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:06.190267 17548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:06.190286 17548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:06.190304 17548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:06.190318 17548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:06.195434 17548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:43935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:06.196398 17548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:06.197329 17548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:06.205883 17554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.205868 17556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:06.205972 17548 server_base.cc:1034] running on GCE node
W20260927 10:49:06.205920 17553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:06.206769 17548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:06.207866 17548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:06.209048 17548 hybrid_clock.cc:648] HybridClock initialized: now 1790506146209011 us; error 44 us; skew 500 ppm
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:06.215370 17548 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:49:06.216332 17548 webserver.cc:460] Webserver started at http://127.9.39.190:40293/ using document root <none> and password file <none>
I20260927 10:49:06.216858 17548 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:06.216913 17548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:06.217078 17548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:06.219261 17548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "2150907f269b4c7289afb0c79d774154"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "a293f201eb02c79ea7362835399657b1"
server_key_iv: "7e5d8cd1c3c9e6007f9dd0eb039722a8"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.219786 17548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "2150907f269b4c7289afb0c79d774154"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "a293f201eb02c79ea7362835399657b1"
server_key_iv: "7e5d8cd1c3c9e6007f9dd0eb039722a8"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.223631 17548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 10:49:06.226068 17563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.227397 17548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:49:06.227552 17548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "2150907f269b4c7289afb0c79d774154"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "a293f201eb02c79ea7362835399657b1"
server_key_iv: "7e5d8cd1c3c9e6007f9dd0eb039722a8"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.227676 17548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:06.242992 17548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:06.246047 17548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:06.246271 17548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:06.253337 17548 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:43935
I20260927 10:49:06.253345 17615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:43935 every 8 connection(s)
I20260927 10:49:06.254397 17548 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:49:06.257117 17616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:06.262491 17616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Bootstrap starting.
I20260927 10:49:06.264464 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 17548
I20260927 10:49:06.264570 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:49:06.264864 9374 external_mini_cluster.cc:1428] Setting key 88b9d82bc128edb48d1c021f13bc7d9b
I20260927 10:49:06.265007 17616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:06.265942 17616 log.cc:824] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:06.268229 17616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: No bootstrap required, opened a new log
I20260927 10:49:06.272668 17616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:06.272956 17616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:06.273042 17616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2150907f269b4c7289afb0c79d774154, State: Initialized, Role: FOLLOWER
I20260927 10:49:06.273526 17616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:06.273648 17616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:49:06.273732 17616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:49:06.273825 17616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:49:06.274907 17616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:06.275280 17616 leader_election.cc:302] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2150907f269b4c7289afb0c79d774154; no voters:
I20260927 10:49:06.275636 17616 leader_election.cc:288] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:49:06.275710 17621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:06.275923 17621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 LEADER]: Becoming Leader. State: Replica: 2150907f269b4c7289afb0c79d774154, State: Running, Role: LEADER
I20260927 10:49:06.276409 17621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:06.276962 17616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:49:06.277608 17621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2150907f269b4c7289afb0c79d774154. Latest consensus state: current_term: 1 leader_uuid: "2150907f269b4c7289afb0c79d774154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } } }
I20260927 10:49:06.277751 17621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:06.278334 17622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2150907f269b4c7289afb0c79d774154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } } }
I20260927 10:49:06.278494 17622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:06.278832 17629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:49:06.280884 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:06.266096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.9.39.190:43935)
0927 10:49:06.266556 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:06.266571 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:06.266605 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:49:06.267474 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:06.267499 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:06.267909 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:06.268276 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:06.269014 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.270540 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.271457 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.272177 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.272334 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:06.274960 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:06.274966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:06.274983 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:06.275015 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:06.277206 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:06.277714 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:06.277720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:06.277726 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:06.277907 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:06.278264 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:06.278268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:06.278270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:06.278850 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:06.279747 (+ 897us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:06.279929 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 10:49:06.280184 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"spinlock_wait_cycles":11520,"thread_start_us":167,"threads_started":1}
I20260927 10:49:06.282290 17629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:49:06.287746 17629 catalog_manager.cc:1347] Generated new cluster ID: 844810065b88452baea19bc043efccdc
I20260927 10:49:06.287817 17629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:49:06.301519 17629 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:49:06.302819 17629 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:49:06.311372 17629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Generated new TSK 0
I20260927 10:49:06.312116 17629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:49:06.345638 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43935
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:06.472455 17642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:06.472775 17642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:06.472827 17642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:06.477322 17642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:06.477396 17642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:06.477496 17642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:49:06.483165 17642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:06.484344 17642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:06.485373 17642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:06.494943 17650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.494943 17648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.494946 17647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:06.494977 17642 server_base.cc:1034] running on GCE node
I20260927 10:49:06.495601 17642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:06.496217 17642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:06.497385 17642 hybrid_clock.cc:648] HybridClock initialized: now 1790506146497351 us; error 53 us; skew 500 ppm
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:06.500445 17642 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:49:06.501315 17642 webserver.cc:460] Webserver started at http://127.9.39.129:46741/ using document root <none> and password file <none>
I20260927 10:49:06.501850 17642 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:06.501909 17642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:06.502163 17642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:06.504233 17642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "5fd266806b1d4d16a22135b7d438f1d0"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "384b84c8cb5c2a2dadc89a1c070040dc"
server_key_iv: "0e722b58f114c645238c00998e3e7c84"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.504797 17642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "5fd266806b1d4d16a22135b7d438f1d0"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "384b84c8cb5c2a2dadc89a1c070040dc"
server_key_iv: "0e722b58f114c645238c00998e3e7c84"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.508493 17642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:49:06.510947 17657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.512209 17642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:06.512341 17642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "5fd266806b1d4d16a22135b7d438f1d0"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "384b84c8cb5c2a2dadc89a1c070040dc"
server_key_iv: "0e722b58f114c645238c00998e3e7c84"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.512450 17642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:06.530745 17642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:06.533964 17642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:06.534178 17642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:06.534782 17642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:06.535604 17642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:06.535673 17642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.535743 17642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:06.535799 17642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.544768 17642 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:41951
I20260927 10:49:06.544798 17770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:41951 every 8 connection(s)
I20260927 10:49:06.545625 17642 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:49:06.555071 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 17642
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:49:06.555207 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:49:06.555522 9374 external_mini_cluster.cc:1428] Setting key 1261aee2e176000787e2b0362d2a6af6
I20260927 10:49:06.559755 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:06.547400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:35061 (local address 127.9.39.190:43935)
0927 10:49:06.547599 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:06.547604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:06.548429 (+ 825us) server_negotiation.cc:407] Connection header received
0927 10:49:06.549710 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:06.549713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:06.549769 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:06.549851 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:06.551034 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.551595 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.552740 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.552982 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.553098 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:06.555853 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:06.555857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:06.555859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:06.555883 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:06.557508 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:06.558123 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:06.558126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:06.558127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:06.558205 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:06.558633 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:06.558635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:06.558636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:06.558858 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:06.558969 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:06.559466 (+ 497us) server_negotiation.cc:299] Negotiation successful
0927 10:49:06.559587 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 10:49:06.560549 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:06.547774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:35061)
0927 10:49:06.548267 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:06.548310 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:06.549493 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:06.550027 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:06.550034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:06.550394 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:06.550886 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:06.550900 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.551735 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.551740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:06.552614 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:06.552623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.553142 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.553146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:06.553324 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:06.553917 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:06.553932 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:06.555686 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:06.557629 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:06.557638 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:06.557656 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:06.558028 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:06.558295 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:06.558315 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:06.558320 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:06.558515 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:06.558969 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:06.558977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:06.559227 (+ 250us) client_negotiation.cc:769] Sending connection context
0927 10:49:06.559414 (+ 187us) client_negotiation.cc:240] Negotiation successful
0927 10:49:06.559747 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":111,"threads_started":1}
I20260927 10:49:06.561905 17771 heartbeater.cc:347] Connected to a master server at 127.9.39.190:43935
I20260927 10:49:06.562201 17771 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:06.562875 17771 heartbeater.cc:511] Master 127.9.39.190:43935 requested a full tablet report, sending...
I20260927 10:49:06.564710 17580 ts_manager.cc:194] Registered new tserver with Master: 5fd266806b1d4d16a22135b7d438f1d0 (127.9.39.129:41951)
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:49:06.566224 17580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:35061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:49:06.582475 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43935
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:06.692190 17776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:06.692458 17776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:06.692504 17776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:06.696671 17776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:06.696743 17776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:06.696805 17776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:49:06.702502 17776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:06.703617 17776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:06.704622 17776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:06.715432 17784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.715427 17782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.715432 17781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:06.715839 17776 server_base.cc:1034] running on GCE node
I20260927 10:49:06.716382 17776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:06.717132 17776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:06.718294 17776 hybrid_clock.cc:648] HybridClock initialized: now 1790506146718258 us; error 53 us; skew 500 ppm
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:06.721117 17776 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:49:06.722010 17776 webserver.cc:460] Webserver started at http://127.9.39.130:37883/ using document root <none> and password file <none>
I20260927 10:49:06.722561 17776 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:06.722621 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:06.722867 17776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:06.724911 17776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "3e9c95c1ec8d4db980d235e3d5552d4c"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "6417843dd00424281307f1b869cf61f0"
server_key_iv: "c024d24533affc2e8aaf73f955ae317b"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.725471 17776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "3e9c95c1ec8d4db980d235e3d5552d4c"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "6417843dd00424281307f1b869cf61f0"
server_key_iv: "c024d24533affc2e8aaf73f955ae317b"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.729120 17776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:49:06.731637 17791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.732788 17776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:06.732916 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3e9c95c1ec8d4db980d235e3d5552d4c"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "6417843dd00424281307f1b869cf61f0"
server_key_iv: "c024d24533affc2e8aaf73f955ae317b"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.733021 17776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:06.746230 17776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:06.749208 17776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:06.749400 17776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:06.749962 17776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:06.750813 17776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:06.750875 17776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.750939 17776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:06.750984 17776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.759593 17776 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:42107
I20260927 10:49:06.759631 17904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:42107 every 8 connection(s)
I20260927 10:49:06.760476 17776 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:49:06.768622 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 17776
I20260927 10:49:06.768784 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:49:06.769181 9374 external_mini_cluster.cc:1428] Setting key 4e3dae17fa2e0e02392ddb9243e54bda
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:49:06.773695 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:06.762382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:48723 (local address 127.9.39.190:43935)
0927 10:49:06.762510 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:06.762514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:06.763550 (+ 1036us) server_negotiation.cc:407] Connection header received
0927 10:49:06.764374 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:06.764378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:06.764443 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:06.764517 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:06.765647 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.766184 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.767324 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.767534 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.767630 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:06.770129 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:06.770133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:06.770136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:06.770159 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:06.771696 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:06.772206 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:06.772211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:06.772213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:06.772294 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:06.772632 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:06.772635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:06.772636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:06.772811 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:06.772921 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:06.773400 (+ 479us) server_negotiation.cc:299] Negotiation successful
0927 10:49:06.773530 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:49:06.774592 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:06.762777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:48723)
0927 10:49:06.763402 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:06.763439 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:06.764163 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:06.764705 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:06.764712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:06.765054 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:06.765521 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:06.765533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.766287 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.766290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:06.767224 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:06.767233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:06.767671 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:06.767674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:06.767804 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:06.768315 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:06.768330 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:06.769981 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:06.771815 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:06.771821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:06.771835 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:06.772107 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:06.772386 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:06.772391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:06.772393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:06.772531 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:06.772919 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:06.772926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:06.773168 (+ 242us) client_negotiation.cc:769] Sending connection context
0927 10:49:06.773347 (+ 179us) client_negotiation.cc:240] Negotiation successful
0927 10:49:06.773667 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":109,"threads_started":1}
I20260927 10:49:06.775768 17905 heartbeater.cc:347] Connected to a master server at 127.9.39.190:43935
I20260927 10:49:06.776034 17905 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:06.776640 17905 heartbeater.cc:511] Master 127.9.39.190:43935 requested a full tablet report, sending...
I20260927 10:49:06.777827 17580 ts_manager.cc:194] Registered new tserver with Master: 3e9c95c1ec8d4db980d235e3d5552d4c (127.9.39.130:42107)
I20260927 10:49:06.778501 17580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:48723
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:49:06.797154 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43935
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:06.906527 17910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:06.906795 17910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:06.906838 17910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:06.911080 17910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:06.911137 17910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:06.911221 17910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:49:06.916520 17910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:06.917562 17910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:06.918561 17910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:06.927872 17915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.927857 17916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:06.927842 17918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:06.928056 17910 server_base.cc:1034] running on GCE node
I20260927 10:49:06.928552 17910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:06.929183 17910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:06.930377 17910 hybrid_clock.cc:648] HybridClock initialized: now 1790506146930347 us; error 47 us; skew 500 ppm
Sep 27 10:49:06 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:06.933465 17910 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:49:06.934264 17910 webserver.cc:460] Webserver started at http://127.9.39.131:37203/ using document root <none> and password file <none>
I20260927 10:49:06.934828 17910 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:06.934891 17910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:06.935237 17910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:06.937383 17910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "030268f7c80740aba25c76fb6832e07d"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "fd01d6004beb811b226f60b273a9d9ee"
server_key_iv: "b91ffae2e2a114a46af4981681f3bdb2"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.937878 17910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "030268f7c80740aba25c76fb6832e07d"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "fd01d6004beb811b226f60b273a9d9ee"
server_key_iv: "b91ffae2e2a114a46af4981681f3bdb2"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.941466 17910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 10:49:06.943802 17925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.945142 17910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:49:06.945297 17910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "030268f7c80740aba25c76fb6832e07d"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "fd01d6004beb811b226f60b273a9d9ee"
server_key_iv: "b91ffae2e2a114a46af4981681f3bdb2"
server_key_version: "encryptionkey@0"
I20260927 10:49:06.945413 17910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:06.993865 17910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:06.996975 17910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:06.997206 17910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:06.997821 17910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:06.998714 17910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:06.998790 17910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:06.998862 17910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:06.998910 17910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.008070 17910 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:39945
I20260927 10:49:07.008087 18038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:39945 every 8 connection(s)
I20260927 10:49:07.008968 17910 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:49:07.014355 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 17910
I20260927 10:49:07.014490 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:49:07.014812 9374 external_mini_cluster.cc:1428] Setting key d72bfc2a61c1ab3108454a985983f3c4
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:49:07.022301 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.010792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:60227 (local address 127.9.39.190:43935)
0927 10:49:07.010930 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.010934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.011970 (+ 1036us) server_negotiation.cc:407] Connection header received
0927 10:49:07.012824 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.012827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.012873 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.012949 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.014115 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.014827 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.016237 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.016484 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.016564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.019183 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:07.019186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:07.019188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:07.019223 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:07.020495 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:07.020988 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:07.020990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:07.020991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:07.021059 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:07.021402 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:07.021404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:07.021405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:07.021562 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:07.021635 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:07.022084 (+ 449us) server_negotiation.cc:299] Negotiation successful
0927 10:49:07.022177 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:49:07.023257 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.011221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:60227)
0927 10:49:07.011848 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.011890 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.012626 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.013115 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.013121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.013487 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.013952 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.013966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.014973 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.014977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.016112 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.016122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.016638 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.016643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.016840 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.017434 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.017451 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.019028 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:07.020604 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:07.020609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:07.020621 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:07.020899 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:07.021169 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:07.021172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:07.021174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:07.021312 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:07.021648 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:07.021653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:07.021878 (+ 225us) client_negotiation.cc:769] Sending connection context
0927 10:49:07.022043 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 10:49:07.022300 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":113,"threads_started":1}
I20260927 10:49:07.024369 18039 heartbeater.cc:347] Connected to a master server at 127.9.39.190:43935
I20260927 10:49:07.024603 18039 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:07.025067 18039 heartbeater.cc:511] Master 127.9.39.190:43935 requested a full tablet report, sending...
I20260927 10:49:07.026046 17580 ts_manager.cc:194] Registered new tserver with Master: 030268f7c80740aba25c76fb6832e07d (127.9.39.131:39945)
I20260927 10:49:07.026629 17580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:60227
I20260927 10:49:07.028714 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab.
Entry for principal oryx/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab.
WARNING: no policy specified for oryx/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab.
Entry for principal oryx/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab.
WARNING: no policy specified for oryx/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab.
Entry for principal oryx/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab.
I20260927 10:49:07.099812 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 17642
I20260927 10:49:07.105408 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 17776
I20260927 10:49:07.110554 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 17910
I20260927 10:49:07.115594 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 17548
I20260927 10:49:07.121543 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43935
--webserver_interface=127.9.39.190
--webserver_port=40293
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:07.230660 18048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:07.230975 18048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:07.231016 18048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:07.235023 18048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:07.235083 18048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:07.235102 18048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:07.235123 18048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:07.235139 18048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:07.240262 18048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:43935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=40293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:07.241235 18048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:07.242154 18048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:07.250586 18054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.250685 18053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.250589 18056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:07.250845 18048 server_base.cc:1034] running on GCE node
I20260927 10:49:07.251498 18048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:07.252530 18048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:07.253690 18048 hybrid_clock.cc:648] HybridClock initialized: now 1790506147253665 us; error 37 us; skew 500 ppm
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506147, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:07.256357 18048 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:49:07.257232 18048 webserver.cc:460] Webserver started at http://127.9.39.190:40293/ using document root <none> and password file <none>
I20260927 10:49:07.257773 18048 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:07.257854 18048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:07.261662 18048 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:07.263777 18063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.264992 18048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:07.265123 18048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "2150907f269b4c7289afb0c79d774154"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "a293f201eb02c79ea7362835399657b1"
server_key_iv: "7e5d8cd1c3c9e6007f9dd0eb039722a8"
server_key_version: "encryptionkey@0"
I20260927 10:49:07.265669 18048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:07.286229 18048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:07.289315 18048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:07.289532 18048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:07.296743 18048 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:43935
I20260927 10:49:07.296765 18115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:43935 every 8 connection(s)
I20260927 10:49:07.297727 18048 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:49:07.298056 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18048
I20260927 10:49:07.298534 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:41951
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:43935
--webserver_port=46741
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:49:07.302274 18116 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:49:07.303948 18116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Bootstrap starting.
I20260927 10:49:07.320041 18116 log.cc:824] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:07.326253 18116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:49:07.326615 18116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Bootstrap complete.
I20260927 10:49:07.330873 18116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:07.331302 18116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2150907f269b4c7289afb0c79d774154, State: Initialized, Role: FOLLOWER
I20260927 10:49:07.331779 18116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:07.331921 18116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:49:07.331990 18116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:49:07.332101 18116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 1 FOLLOWER]: Advancing to term 2
I20260927 10:49:07.334125 18116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:07.334452 18116 leader_election.cc:302] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2150907f269b4c7289afb0c79d774154; no voters:
I20260927 10:49:07.334842 18116 leader_election.cc:288] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 10:49:07.334906 18120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 10:49:07.335110 18120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [term 2 LEADER]: Becoming Leader. State: Replica: 2150907f269b4c7289afb0c79d774154, State: Running, Role: LEADER
I20260927 10:49:07.335450 18120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } }
I20260927 10:49:07.336061 18116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:49:07.336715 18121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2150907f269b4c7289afb0c79d774154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } } }
I20260927 10:49:07.337109 18121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:07.337008 18122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2150907f269b4c7289afb0c79d774154. Latest consensus state: current_term: 2 leader_uuid: "2150907f269b4c7289afb0c79d774154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2150907f269b4c7289afb0c79d774154" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 43935 } } }
I20260927 10:49:07.337412 18122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:07.337677 18129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:49:07.340889 18129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:49:07.341821 18129 catalog_manager.cc:1259] Loaded cluster ID: 844810065b88452baea19bc043efccdc
I20260927 10:49:07.341894 18129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:49:07.344211 18129 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:49:07.344841 18129 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2150907f269b4c7289afb0c79d774154: Loaded TSK: 0
I20260927 10:49:07.346027 18129 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 10:49:07.412040 18118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:07.412349 18118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:07.412422 18118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:07.416494 18118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:07.416568 18118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:07.416658 18118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:49:07.421875 18118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.129.keytab
--principal=oryx/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:41951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=46741
--tserver_master_addrs=127.9.39.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:07.422948 18118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:07.423921 18118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:07.434242 18142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.434309 18143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.434455 18145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:07.434746 18118 server_base.cc:1034] running on GCE node
I20260927 10:49:07.435189 18118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:07.435935 18118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:07.437129 18118 hybrid_clock.cc:648] HybridClock initialized: now 1790506147437087 us; error 45 us; skew 500 ppm
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506147, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:07.440501 18118 init.cc:377] Logged in from keytab as oryx/127.9.39.129@KRBTEST.COM (short username oryx)
I20260927 10:49:07.441368 18118 webserver.cc:460] Webserver started at http://127.9.39.129:46741/ using document root <none> and password file <none>
I20260927 10:49:07.441910 18118 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:07.441982 18118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:07.445909 18118 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:07.448268 18152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.449538 18118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:49:07.449669 18118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "5fd266806b1d4d16a22135b7d438f1d0"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "384b84c8cb5c2a2dadc89a1c070040dc"
server_key_iv: "0e722b58f114c645238c00998e3e7c84"
server_key_version: "encryptionkey@0"
I20260927 10:49:07.450204 18118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:07.466318 18118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:07.469841 18118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:07.470057 18118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:07.470664 18118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:07.471611 18118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:07.471678 18118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.471750 18118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:07.471791 18118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.481774 18118 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:41951
I20260927 10:49:07.481797 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:41951 every 8 connection(s)
I20260927 10:49:07.482915 18118 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:49:07.485028 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18118
I20260927 10:49:07.485560 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:42107
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:43935
--webserver_port=37883
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 10:49:07.496042 18268 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.485063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:41689 (local address 127.9.39.190:43935)
0927 10:49:07.485553 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.485575 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.486141 (+ 566us) server_negotiation.cc:407] Connection header received
0927 10:49:07.487263 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.487280 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.487693 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.488091 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.489089 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.490034 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.491216 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.491669 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.491839 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.495397 (+ 3558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:41689: BlockingRecv error: recv got EOF from 127.9.39.129:41689 (error 108)
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":112,"threads_started":1}
I20260927 10:49:07.496189 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.485260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:41689)
0927 10:49:07.485959 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.486009 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.486858 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.488138 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.488144 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.488485 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.488923 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.488936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.490167 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.490171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.491096 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.491105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.491828 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.491832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.492015 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.492629 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.492644 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.495224 (+ 2580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":126,"threads_started":1}
W20260927 10:49:07.496721 18266 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:43935 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.39.190:43935: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.502429 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.497190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:39003)
0927 10:49:07.497346 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.497358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.497421 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.497625 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.497627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.497780 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.497897 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.497903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.498549 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.498552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.499459 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.499468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.499915 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.499918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.500021 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.500420 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.500436 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.502288 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260927 10:49:07.502511 18268 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.497274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:39003 (local address 127.9.39.190:43935)
0927 10:49:07.497387 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.497390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.497401 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:49:07.497495 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.497497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.497528 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.497591 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.497994 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.498457 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.499562 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.499814 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.499891 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.502397 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:39003: BlockingRecv error: recv got EOF from 127.9.39.129:39003 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.508037 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.503153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:42531)
0927 10:49:07.503374 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.503388 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.503488 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.503732 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.503735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.503888 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.503952 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.503956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.504645 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.504649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.505154 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.505160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.505592 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.505596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.505677 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.506134 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.506147 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.507902 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260927 10:49:07.508121 18268 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.503245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:42531 (local address 127.9.39.190:43935)
0927 10:49:07.503350 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.503354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.503451 (+ 97us) server_negotiation.cc:407] Connection header received
0927 10:49:07.503585 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.503588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.503633 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.503721 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.504057 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.504535 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.505270 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.505480 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.505581 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.508006 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:42531: BlockingRecv error: recv got EOF from 127.9.39.129:42531 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 10:49:07.508389 18266 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:49:07.598619 18270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:07.598940 18270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:07.598982 18270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:07.602970 18270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:07.603026 18270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:07.603089 18270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:49:07.608242 18270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.130.keytab
--principal=oryx/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:42107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=37883
--tserver_master_addrs=127.9.39.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:07.609244 18270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:07.610216 18270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:07.620126 18278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.620160 18275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:07.620270 18270 server_base.cc:1034] running on GCE node
W20260927 10:49:07.620138 18276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:07.620764 18270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:07.621378 18270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:07.622540 18270 hybrid_clock.cc:648] HybridClock initialized: now 1790506147622525 us; error 28 us; skew 500 ppm
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506147, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:07.625229 18270 init.cc:377] Logged in from keytab as oryx/127.9.39.130@KRBTEST.COM (short username oryx)
I20260927 10:49:07.626060 18270 webserver.cc:460] Webserver started at http://127.9.39.130:37883/ using document root <none> and password file <none>
I20260927 10:49:07.626593 18270 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:07.626727 18270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:07.630534 18270 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260927 10:49:07.632514 18285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.633699 18270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:07.633826 18270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3e9c95c1ec8d4db980d235e3d5552d4c"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "6417843dd00424281307f1b869cf61f0"
server_key_iv: "c024d24533affc2e8aaf73f955ae317b"
server_key_version: "encryptionkey@0"
I20260927 10:49:07.634341 18270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:07.650976 18270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:07.653772 18270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:07.654006 18270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:07.654597 18270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:07.655421 18270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:07.655493 18270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.655560 18270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:07.655610 18270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.664441 18270 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:42107
I20260927 10:49:07.664466 18398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:42107 every 8 connection(s)
I20260927 10:49:07.665380 18270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:49:07.672415 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18270
I20260927 10:49:07.672919 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:39945
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:43935
--webserver_port=37203
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 10:49:07.676585 18268 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.667208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:43479 (local address 127.9.39.190:43935)
0927 10:49:07.667344 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.667347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.668370 (+ 1023us) server_negotiation.cc:407] Connection header received
0927 10:49:07.669245 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.669248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.669296 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.669369 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.670541 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.671124 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.672215 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.672423 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.672554 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.676468 (+ 3914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:43479: BlockingRecv error: recv got EOF from 127.9.39.130:43479 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:49:07.677163 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.667559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:43479)
0927 10:49:07.668182 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.668228 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.669054 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.669527 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.669533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.669939 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.670415 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.670428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.671233 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.671236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.672080 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.672089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.672616 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.672620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.672841 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.673576 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.673601 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.676357 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":122,"threads_started":1}
W20260927 10:49:07.677619 18399 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:43935 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.39.190:43935: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.683149 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.678094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:42181)
0927 10:49:07.678225 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.678242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.678318 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.678577 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.678579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.678792 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.678875 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.678880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.679576 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.679583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.680115 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.680122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.680560 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.680564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.680661 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.681160 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.681170 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.683018 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 10:49:07.683252 18268 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.678168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:42181 (local address 127.9.39.190:43935)
0927 10:49:07.678301 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.678304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.678317 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:07.678426 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.678429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.678476 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.678563 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.678984 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.679472 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.680239 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.680452 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.680560 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.683124 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:42181: BlockingRecv error: recv got EOF from 127.9.39.130:42181 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.689067 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.683760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:43911)
0927 10:49:07.683900 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.683913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.683973 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.684177 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.684180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.684333 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.684414 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.684419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.685066 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.685069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.686016 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.686026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.686478 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.686482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.686587 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.687097 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.687111 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.688946 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260927 10:49:07.689174 18268 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.683847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:43911 (local address 127.9.39.190:43935)
0927 10:49:07.683956 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.683959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.683970 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:49:07.684047 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.684049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.684079 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.684142 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.684511 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.684972 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.686131 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.686346 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.686426 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.689057 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:43911: BlockingRecv error: recv got EOF from 127.9.39.130:43911 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 10:49:07.689266 18399 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:49:07.788736 18402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:07.789023 18402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:07.789103 18402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:07.793182 18402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:07.793267 18402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:07.793354 18402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:49:07.798390 18402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/krb5kdc/oryx_127.9.39.131.keytab
--principal=oryx/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:39945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=37203
--tserver_master_addrs=127.9.39.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:07.799396 18402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:07.800391 18402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:07.809715 18408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.809765 18410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:07.809724 18407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:07.810016 18402 server_base.cc:1034] running on GCE node
I20260927 10:49:07.810431 18402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:07.811061 18402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:07.812222 18402 hybrid_clock.cc:648] HybridClock initialized: now 1790506147812209 us; error 36 us; skew 500 ppm
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506147, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:07.814873 18402 init.cc:377] Logged in from keytab as oryx/127.9.39.131@KRBTEST.COM (short username oryx)
I20260927 10:49:07.815752 18402 webserver.cc:460] Webserver started at http://127.9.39.131:37203/ using document root <none> and password file <none>
I20260927 10:49:07.816262 18402 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:07.816372 18402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:07.820425 18402 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:49:07.822511 18417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.823709 18402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:07.823860 18402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "030268f7c80740aba25c76fb6832e07d"
format_stamp: "Formatted at 2026-09-27 10:49:06 on dist-test-slave-p9q6"
server_key: "fd01d6004beb811b226f60b273a9d9ee"
server_key_iv: "b91ffae2e2a114a46af4981681f3bdb2"
server_key_version: "encryptionkey@0"
I20260927 10:49:07.824379 18402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:07.843868 18402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:07.847153 18402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:07.847366 18402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:07.847954 18402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:07.848690 18402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:07.848737 18402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.848774 18402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:07.848794 18402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:07.857573 18402 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:39945
I20260927 10:49:07.857640 18530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:39945 every 8 connection(s)
I20260927 10:49:07.858588 18402 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:49:07.859663 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18402
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.869614 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.860367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.9.39.190:43935)
0927 10:49:07.860498 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.860502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.860783 (+ 281us) server_negotiation.cc:407] Connection header received
0927 10:49:07.860955 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.860958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.861010 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.861112 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.862020 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.862544 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.863741 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.863945 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.864024 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.865612 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:07.865617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:07.865628 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:07.865653 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:07.867413 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:07.867868 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:07.867878 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:07.867882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:07.867990 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:07.868291 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:07.868295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:07.868297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:07.868743 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:07.869032 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:07.869215 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 10:49:07.869464 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260927 10:49:07.870246 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.860714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:46983 (local address 127.9.39.190:43935)
0927 10:49:07.860935 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.860939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.861826 (+ 887us) server_negotiation.cc:407] Connection header received
0927 10:49:07.862766 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.862769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.862818 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.862886 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.864249 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.864885 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.866380 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.866743 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.866846 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.870159 (+ 3313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:46983: BlockingRecv error: recv got EOF from 127.9.39.131:46983 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260927 10:49:07.870970 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.860834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:46983)
0927 10:49:07.861653 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.861723 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.862521 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.863060 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.863069 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.863491 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.864114 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.864134 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.865007 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.865013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.866264 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.866273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.866875 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.866878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.867020 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.867531 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.867546 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.870060 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":143,"threads_started":1}
W20260927 10:49:07.871483 18531 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:43935 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.39.190:43935: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.876467 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.871872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:59991)
0927 10:49:07.871984 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.871999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.872073 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.872478 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.872480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.872637 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.872772 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.872778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.873371 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.873373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.873870 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.873877 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.874258 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.874261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.874320 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.874751 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.874762 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.876342 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 10:49:07.876545 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.872166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:59991 (local address 127.9.39.190:43935)
0927 10:49:07.872280 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.872283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.872311 (+ 28us) server_negotiation.cc:407] Connection header received
0927 10:49:07.872343 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.872345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.872380 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.872451 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.872854 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.873292 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.873960 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.874176 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.874247 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.876444 (+ 2197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:59991: BlockingRecv error: recv got EOF from 127.9.39.131:59991 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:07 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:07.881314 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:07.877009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:49697)
0927 10:49:07.877103 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:07.877114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:07.877201 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:07.877422 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:07.877425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:07.877565 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:07.877625 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.877630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.878297 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.878302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.878862 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:07.878870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.879285 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.879288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:07.879365 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.879708 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:07.879719 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:07.881193 (+ 1474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 10:49:07.881388 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:07.877096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:49697 (local address 127.9.39.190:43935)
0927 10:49:07.877237 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:07.877243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:07.877256 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:07.877287 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:07.877290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:07.877328 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:07.877408 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:07.877729 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.878189 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.878970 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:07.879182 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:07.879286 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:07.881290 (+ 2004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:49697: BlockingRecv error: recv got EOF from 127.9.39.131:49697 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260927 10:49:07.881527 18531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 10:49:08 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:08 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:08.514225 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:08.508963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:51713)
0927 10:49:08.509211 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:08.509229 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:08.509330 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:08.509543 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:08.509545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:08.509714 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:08.509820 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:08.509825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.510552 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.510555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:08.511251 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:08.511260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.511805 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.511809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:08.511915 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:08.512421 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:08.512449 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:08.514114 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":63,"threads_started":1}
W20260927 10:49:08.514410 18537 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:08.509072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:51713 (local address 127.9.39.190:43935)
0927 10:49:08.509333 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:08.509337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:08.509350 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:08.509389 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:08.509391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:08.509431 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:08.509505 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:08.509917 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.510443 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.511350 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.511626 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.511762 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:08.514245 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:51713: BlockingRecv error: recv got EOF from 127.9.39.129:51713 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":98,"threads_started":1}
Sep 27 10:49:08 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:08 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:08.695660 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:08.689861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:54279)
0927 10:49:08.690123 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:08.690142 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:08.690240 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:08.690596 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:08.690598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:08.690793 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:08.690888 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:08.690893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.691643 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.691646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:08.692217 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:08.692225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.692679 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.692682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:08.692748 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:08.693312 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:08.693326 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:08.695517 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
W20260927 10:49:08.695855 18537 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:08.689987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:54279 (local address 127.9.39.190:43935)
0927 10:49:08.690142 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:08.690146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:08.690213 (+ 67us) server_negotiation.cc:407] Connection header received
0927 10:49:08.690379 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:08.690382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:08.690439 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:08.690553 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:08.691008 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.691541 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.692315 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.692536 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.692639 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:08.695682 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:54279: BlockingRecv error: recv got EOF from 127.9.39.130:54279 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 10:49:08 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:08 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:08.888010 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:08.882149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:34215)
0927 10:49:08.882404 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:08.882419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:08.882528 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:08.882775 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:08.882778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:08.882960 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:08.883049 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:08.883054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.883826 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.883829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:08.884724 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:08.884734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.885266 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.885268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:08.885328 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:08.885952 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:08.885968 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:08.887889 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
W20260927 10:49:08.888290 18537 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:08.882239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34215 (local address 127.9.39.190:43935)
0927 10:49:08.882373 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:08.882382 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:08.882506 (+ 124us) server_negotiation.cc:407] Connection header received
0927 10:49:08.882610 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:08.882613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:08.882657 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:08.882752 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:08.883144 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.883702 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.884864 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:08.885164 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:08.885247 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:08.888025 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:34215: BlockingRecv error: recv got EOF from 127.9.39.131:34215 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 27 10:49:09 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:09 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:09.520634 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:09.515073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:40527)
0927 10:49:09.515401 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:09.515415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:09.515520 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:09.515767 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:09.515769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:09.515940 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:09.516039 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:09.516044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.516777 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.516779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:09.517341 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:09.517349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.517784 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.517787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:09.517865 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:09.518502 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:09.518515 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:09.520507 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":88,"threads_started":1}
W20260927 10:49:09.520766 18541 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:09.515193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:40527 (local address 127.9.39.190:43935)
0927 10:49:09.515433 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:09.515436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:09.515477 (+ 41us) server_negotiation.cc:407] Connection header received
0927 10:49:09.515605 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:09.515607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:09.515650 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:09.515733 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:09.516133 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.516657 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.517454 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.517682 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.517759 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:09.520635 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:40527: BlockingRecv error: recv got EOF from 127.9.39.129:40527 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
Sep 27 10:49:09 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:09 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:09.702733 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:09.696533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:53075)
0927 10:49:09.696819 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:09.696833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:09.696933 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:09.697183 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:09.697185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:09.697356 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:09.697450 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:09.697455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.698391 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.698395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:09.699330 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:09.699340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.699836 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.699840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:09.699931 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:09.700414 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:09.700426 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:09.702492 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":117,"threads_started":1}
W20260927 10:49:09.702804 18541 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:09.696640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:53075 (local address 127.9.39.190:43935)
0927 10:49:09.696779 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:09.696783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:09.696904 (+ 121us) server_negotiation.cc:407] Connection header received
0927 10:49:09.697027 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:09.697029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:09.697073 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:09.697136 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:09.697548 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.698219 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.699463 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.699715 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.699830 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:09.702653 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:53075: BlockingRecv error: recv got EOF from 127.9.39.130:53075 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 27 10:49:09 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:09 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:09.894315 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:09.888827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:38207)
0927 10:49:09.889183 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:09.889196 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:09.889335 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:09.889638 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:09.889643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:09.889879 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:09.890005 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:09.890011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.890776 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.890779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:09.891350 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:09.891358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.891783 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.891785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:09.891848 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:09.892428 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:09.892440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:09.894179 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":102,"threads_started":1}
W20260927 10:49:09.894479 18541 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:09.888973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:38207 (local address 127.9.39.190:43935)
0927 10:49:09.889124 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:09.889128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:09.889307 (+ 179us) server_negotiation.cc:407] Connection header received
0927 10:49:09.889445 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:09.889448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:09.889506 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:09.889589 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:09.890109 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.890628 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.891459 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:09.891679 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:09.891758 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:09.894323 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:38207: BlockingRecv error: recv got EOF from 127.9.39.131:38207 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 27 10:49:10 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:10 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:10.529099 18544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:10.521531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:41593)
0927 10:49:10.521858 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:10.521875 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:10.522010 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:10.522383 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:10.522386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:10.522587 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:10.522704 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:10.522736 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.523783 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.523787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:10.524743 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:10.524754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.525617 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.525621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:10.525753 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:10.526385 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:10.526418 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:10.528914 (+ 2496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":72,"threads_started":1}
W20260927 10:49:10.529163 18545 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:10.521604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:41593 (local address 127.9.39.190:43935)
0927 10:49:10.521856 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:10.521863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:10.521935 (+ 72us) server_negotiation.cc:407] Connection header received
0927 10:49:10.522151 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:10.522155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:10.522207 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:10.522323 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:10.522837 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.523643 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.524973 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.525315 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.525428 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:10.529033 (+ 3605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:41593: BlockingRecv error: recv got EOF from 127.9.39.129:41593 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":101,"threads_started":1}
Sep 27 10:49:10 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:10 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:10.708979 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:10.703541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:55417)
0927 10:49:10.703848 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:10.703872 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:10.704005 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:10.704286 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:10.704289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:10.704491 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:10.704581 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:10.704586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.705361 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.705366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:10.705949 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:10.705957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.706425 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.706428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:10.706516 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:10.707100 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:10.707111 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:10.708865 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
W20260927 10:49:10.709120 18545 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:10.703652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:55417 (local address 127.9.39.190:43935)
0927 10:49:10.703824 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:10.703828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:10.703946 (+ 118us) server_negotiation.cc:407] Connection header received
0927 10:49:10.704107 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:10.704110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:10.704168 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:10.704259 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:10.704703 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.705247 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.706069 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.706313 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.706422 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:10.709003 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:55417: BlockingRecv error: recv got EOF from 127.9.39.130:55417 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 27 10:49:10 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:10 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:10.900931 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:10.895305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:53899)
0927 10:49:10.895592 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:10.895611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:10.895743 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:10.896023 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:10.896026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:10.896223 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:10.896312 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:10.896317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.897062 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.897067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:10.897637 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:10.897645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.898101 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.898104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:10.898188 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:10.898676 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:10.898688 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:10.900763 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
W20260927 10:49:10.900983 18545 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:10.895395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:53899 (local address 127.9.39.190:43935)
0927 10:49:10.895597 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:10.895601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:10.895709 (+ 108us) server_negotiation.cc:407] Connection header received
0927 10:49:10.895832 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:10.895835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:10.895894 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:10.896000 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:10.896428 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.896931 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.897756 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:10.897989 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:10.898099 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:10.900882 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:53899: BlockingRecv error: recv got EOF from 127.9.39.131:53899 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Sep 27 10:49:11 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:11 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:11.539966 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:11.530015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:51471)
0927 10:49:11.530274 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:11.530288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:11.530378 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:11.530658 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:11.530660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:11.530844 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:11.531002 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:11.531008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.531788 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.531790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:11.532350 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:11.532356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.532820 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.532824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:11.532911 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:11.537574 (+ 4663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:11.537587 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:11.539795 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
W20260927 10:49:11.540094 18549 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:11.530071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:51471 (local address 127.9.39.190:43935)
0927 10:49:11.530311 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:11.530315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:11.530331 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:49:11.530493 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:11.530495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:11.530543 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:11.530623 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:11.531109 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.531693 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.532442 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.532686 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.532794 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:11.539970 (+ 7176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:51471: BlockingRecv error: recv got EOF from 127.9.39.129:51471 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":58,"threads_started":1}
Sep 27 10:49:11 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:11 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:11.715040 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:11.709843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:38493)
0927 10:49:11.710121 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:11.710138 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:11.710255 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:11.710531 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:11.710533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:11.710735 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:11.710830 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:11.710836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.711551 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.711553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:11.712096 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:11.712103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.712487 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.712490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:11.712547 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:11.713015 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:11.713026 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:11.714902 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
W20260927 10:49:11.715119 18549 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:11.710000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:38493 (local address 127.9.39.190:43935)
0927 10:49:11.710162 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:11.710166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:11.710203 (+ 37us) server_negotiation.cc:407] Connection header received
0927 10:49:11.710341 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:11.710344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:11.710386 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:11.710453 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:11.710931 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.711444 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.712189 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.712392 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.712464 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:11.715029 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:38493: BlockingRecv error: recv got EOF from 127.9.39.130:38493 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 27 10:49:11 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:11 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:11.907788 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:11.901794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:59691)
0927 10:49:11.902146 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:11.902166 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:11.902308 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:11.902590 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:11.902593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:11.902815 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:11.902909 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:11.902914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.903697 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.903701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:11.904251 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:11.904258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.904735 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.904739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:11.904826 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:11.905430 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:11.905450 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:11.907676 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":104,"threads_started":1}
W20260927 10:49:11.907970 18549 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:11.901955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:59691 (local address 127.9.39.190:43935)
0927 10:49:11.902125 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:11.902130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:11.902255 (+ 125us) server_negotiation.cc:407] Connection header received
0927 10:49:11.902411 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:11.902415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:11.902475 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:11.902565 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:11.903043 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.903565 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.904366 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:11.904598 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:11.904732 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:11.907820 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:59691: BlockingRecv error: recv got EOF from 127.9.39.131:59691 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 27 10:49:12 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:12 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:12.546231 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:12.540783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:57657)
0927 10:49:12.541078 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:12.541092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:12.541181 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:12.541428 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:12.541431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:12.541611 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:12.541702 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:12.541706 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.542403 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.542406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:12.543057 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:12.543066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.543467 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.543469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:12.543528 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:12.544069 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:12.544082 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:12.546066 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":85,"threads_started":1}
W20260927 10:49:12.546303 18553 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:12.540889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:57657 (local address 127.9.39.190:43935)
0927 10:49:12.541117 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:12.541121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:12.541139 (+ 18us) server_negotiation.cc:407] Connection header received
0927 10:49:12.541271 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:12.541273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:12.541314 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:12.541390 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:12.541796 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.542315 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.543155 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.543375 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.543450 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:12.546184 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:57657: BlockingRecv error: recv got EOF from 127.9.39.129:57657 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":56,"threads_started":1}
Sep 27 10:49:12 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:12 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:12.721853 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:12.716021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:43723)
0927 10:49:12.716370 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:12.716386 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:12.716536 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:12.716826 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:12.716829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:12.717058 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:12.717149 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:12.717154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.717907 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.717913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:12.718476 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:12.718484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.719040 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.719044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:12.719153 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:12.719763 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:12.719775 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:12.721684 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":141,"threads_started":1}
W20260927 10:49:12.721910 18553 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:12.716232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:43723 (local address 127.9.39.190:43935)
0927 10:49:12.716375 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:12.716382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:12.716485 (+ 103us) server_negotiation.cc:407] Connection header received
0927 10:49:12.716651 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:12.716654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:12.716704 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:12.716790 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:12.717266 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.717797 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.718616 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.718903 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.719022 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:12.721805 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:43723: BlockingRecv error: recv got EOF from 127.9.39.130:43723 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 27 10:49:12 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:12 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:12.914132 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:12.908610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:39011)
0927 10:49:12.908905 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:12.908921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:12.909020 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:12.909252 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:12.909254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:12.909438 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:12.909531 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:12.909536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.910275 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.910277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:12.910937 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:12.910945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.911373 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.911376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:12.911451 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:12.912157 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:12.912168 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:12.914006 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":121,"threads_started":1}
W20260927 10:49:12.914283 18553 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:12.908678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:39011 (local address 127.9.39.190:43935)
0927 10:49:12.908829 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:12.908835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:12.908983 (+ 148us) server_negotiation.cc:407] Connection header received
0927 10:49:12.909105 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:12.909107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:12.909155 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:12.909219 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:12.909633 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.910170 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.911051 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:12.911267 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:12.911354 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:12.914156 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:39011: BlockingRecv error: recv got EOF from 127.9.39.131:39011 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 10:49:13 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:13 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:13.552438 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:13.547014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:44515)
0927 10:49:13.547274 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:13.547287 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:13.547390 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:13.547625 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:13.547628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:13.547807 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:13.547899 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:13.547904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.548641 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.548644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:13.549200 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:13.549207 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.549661 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.549664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:13.549732 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:13.550366 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:13.550378 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:13.552282 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
W20260927 10:49:13.552608 18557 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:13.547044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:44515 (local address 127.9.39.190:43935)
0927 10:49:13.547300 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:13.547304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:13.547363 (+ 59us) server_negotiation.cc:407] Connection header received
0927 10:49:13.547471 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:13.547473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:13.547515 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:13.547593 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:13.548000 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.548524 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.549323 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.549554 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.549633 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:13.552439 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:44515: BlockingRecv error: recv got EOF from 127.9.39.129:44515 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":67,"threads_started":1}
Sep 27 10:49:13 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:13 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:13.728163 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:13.722621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:46775)
0927 10:49:13.722984 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:13.722997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:13.723093 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:13.723376 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:13.723379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:13.723568 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:13.723677 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:13.723682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.724395 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.724398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:13.724971 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:13.724980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.725424 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.725426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:13.725490 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:13.726005 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:13.726016 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:13.728048 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
W20260927 10:49:13.728307 18557 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:13.722774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:46775 (local address 127.9.39.190:43935)
0927 10:49:13.722931 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:13.722936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:13.723066 (+ 130us) server_negotiation.cc:407] Connection header received
0927 10:49:13.723183 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:13.723185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:13.723227 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:13.723297 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:13.723779 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.724284 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.725099 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.725316 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.725403 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:13.728191 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:46775: BlockingRecv error: recv got EOF from 127.9.39.130:46775 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 27 10:49:13 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:13 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:13.920526 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:13.915041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:40663)
0927 10:49:13.915334 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:13.915354 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:13.915493 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:13.915783 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:13.915786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:13.916004 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:13.916093 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:13.916099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.916853 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.916856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:13.917424 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:13.917431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.917893 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.917897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:13.917980 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:13.918503 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:13.918515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:13.920414 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
W20260927 10:49:13.920718 18557 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:13.915095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:40663 (local address 127.9.39.190:43935)
0927 10:49:13.915280 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:13.915285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:13.915427 (+ 142us) server_negotiation.cc:407] Connection header received
0927 10:49:13.915598 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:13.915603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:13.915656 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:13.915761 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:13.916212 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.916737 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.917546 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:13.917765 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:13.917891 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:13.920588 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:40663: BlockingRecv error: recv got EOF from 127.9.39.131:40663 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 27 10:49:14 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:14 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:14.558935 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:14.553252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:54437)
0927 10:49:14.553563 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:14.553632 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:14.553742 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:14.554014 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:14.554016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:14.554186 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:14.554272 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:14.554277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.555096 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.555099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:14.555617 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:14.555624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.556005 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.556007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:14.556068 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:14.556639 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:14.556655 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:14.558816 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
W20260927 10:49:14.559080 18561 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:14.553301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:54437 (local address 127.9.39.190:43935)
0927 10:49:14.553573 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:14.553577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:14.553697 (+ 120us) server_negotiation.cc:407] Connection header received
0927 10:49:14.553847 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:14.553850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:14.553902 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:14.553982 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:14.554364 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.555008 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.555709 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.555911 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.555983 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:14.558947 (+ 2964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:54437: BlockingRecv error: recv got EOF from 127.9.39.129:54437 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
Sep 27 10:49:14 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:14 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:14.734208 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:14.729050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:47615)
0927 10:49:14.729397 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:14.729412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:14.729544 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:14.729869 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:14.729872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:14.730124 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:14.730219 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:14.730225 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.730940 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.730942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:14.731493 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:14.731500 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.731885 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.731888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:14.731953 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:14.732453 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:14.732463 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:14.734095 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":97,"threads_started":1}
W20260927 10:49:14.734331 18561 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:14.729085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:47615 (local address 127.9.39.190:43935)
0927 10:49:14.729235 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:14.729242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:14.729495 (+ 253us) server_negotiation.cc:407] Connection header received
0927 10:49:14.729685 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:14.729688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:14.729741 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:14.729827 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:14.730320 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.730855 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.731588 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.731794 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.731870 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:14.734219 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:47615: BlockingRecv error: recv got EOF from 127.9.39.130:47615 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 10:49:14 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:14 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:14.927070 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:14.921326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:55113)
0927 10:49:14.921722 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:14.921739 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:14.921887 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:14.922191 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:14.922194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:14.922410 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:14.922499 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:14.922505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.923314 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.923319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:14.923872 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:14.923880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.924345 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.924348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:14.924444 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:14.925013 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:14.925025 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:14.926890 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":95,"threads_started":1}
W20260927 10:49:14.927105 18561 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:14.921478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:55113 (local address 127.9.39.190:43935)
0927 10:49:14.921649 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:14.921654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:14.921826 (+ 172us) server_negotiation.cc:407] Connection header received
0927 10:49:14.922025 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:14.922028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:14.922082 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:14.922166 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:14.922617 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.923202 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.924012 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:14.924235 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:14.924343 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:14.927008 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:55113: BlockingRecv error: recv got EOF from 127.9.39.131:55113 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 27 10:49:15 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:15 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:15.565569 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:15.559756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:51513)
0927 10:49:15.560062 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:15.560075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:15.560162 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:15.560429 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:15.560431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:15.560602 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:15.560699 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:15.560704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.561475 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.561478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:15.562039 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:15.562046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.562504 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.562507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:15.562565 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:15.563209 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:15.563221 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:15.565438 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":99,"threads_started":1}
W20260927 10:49:15.565732 18565 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:15.559861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:51513 (local address 127.9.39.190:43935)
0927 10:49:15.560131 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:15.560135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:15.560150 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:49:15.560242 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:15.560244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:15.560286 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:15.560389 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:15.560814 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.561372 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.562157 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.562400 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.562492 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:15.565573 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:51513: BlockingRecv error: recv got EOF from 127.9.39.129:51513 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
Sep 27 10:49:15 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:15 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:15.740958 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:15.735165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:41185)
0927 10:49:15.735614 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:15.735650 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:15.735783 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:15.736025 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:15.736028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:15.736206 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:15.736295 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:15.736300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.737187 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.737189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:15.737813 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:15.737821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.738247 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.738251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:15.738336 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:15.738932 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:15.738952 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:15.740765 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":146,"threads_started":1}
W20260927 10:49:15.740975 18565 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:15.735225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:41185 (local address 127.9.39.190:43935)
0927 10:49:15.735415 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:15.735420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:15.735725 (+ 305us) server_negotiation.cc:407] Connection header received
0927 10:49:15.735848 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:15.735850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:15.735894 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:15.735965 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:15.736393 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.737092 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.737916 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.738123 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.738243 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:15.740880 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:41185: BlockingRecv error: recv got EOF from 127.9.39.130:41185 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 27 10:49:15 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:15 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:15.933317 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:15.927802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:39613)
0927 10:49:15.928125 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:15.928138 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:15.928268 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:15.928561 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:15.928564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:15.928781 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:15.928941 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:15.928948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.929811 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.929814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:15.930358 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:15.930366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.930928 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.930931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:15.930992 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:15.931429 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:15.931440 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:15.933211 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":129,"threads_started":1}
W20260927 10:49:15.933449 18565 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:15.927888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:39613 (local address 127.9.39.190:43935)
0927 10:49:15.928053 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:15.928060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:15.928212 (+ 152us) server_negotiation.cc:407] Connection header received
0927 10:49:15.928370 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:15.928373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:15.928431 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:15.928510 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:15.929082 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.929718 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.930454 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:15.930656 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:15.930778 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:15.933332 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:39613: BlockingRecv error: recv got EOF from 127.9.39.131:39613 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 27 10:49:16 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:16 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:16.576304 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:16.566788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:41331)
0927 10:49:16.567126 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:16.567140 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:16.567232 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:16.567430 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:16.567432 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:16.567599 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:16.567688 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:16.567693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.568406 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.568409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:16.568939 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:16.568946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.569550 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.569553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:16.569619 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:16.574198 (+ 4579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:16.574210 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:16.576142 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":93,"threads_started":1}
W20260927 10:49:16.576391 18568 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:16.566817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:41331 (local address 127.9.39.190:43935)
0927 10:49:16.567080 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:16.567084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:16.567205 (+ 121us) server_negotiation.cc:407] Connection header received
0927 10:49:16.567293 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:16.567295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:16.567337 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:16.567420 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:16.567776 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.568298 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.569132 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.569444 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.569532 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:16.576258 (+ 6726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:41331: BlockingRecv error: recv got EOF from 127.9.39.129:41331 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
Sep 27 10:49:16 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:16 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:16.747058 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:16.741661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:34911)
0927 10:49:16.741936 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:16.741949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:16.742041 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:16.742255 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:16.742257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:16.742431 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:16.742522 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:16.742527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.743340 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.743342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:16.743923 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:16.743931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.744331 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.744334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:16.744392 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:16.744945 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:16.744974 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:16.746855 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
W20260927 10:49:16.747184 18568 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:16.741727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:34911 (local address 127.9.39.190:43935)
0927 10:49:16.741856 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:16.741859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:16.742010 (+ 151us) server_negotiation.cc:407] Connection header received
0927 10:49:16.742114 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:16.742116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:16.742158 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:16.742224 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:16.742727 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.743250 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.744031 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.744242 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.744325 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:16.747048 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:34911: BlockingRecv error: recv got EOF from 127.9.39.130:34911 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 10:49:16 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:16 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:16.939587 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:16.934007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:50921)
0927 10:49:16.934309 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:16.934327 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:16.934449 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:16.934726 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:16.934729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:16.934948 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:16.935035 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:16.935040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.935753 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.935760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:16.936375 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:16.936383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.936822 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.936825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:16.936943 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:16.937499 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:16.937511 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:16.939461 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":82,"threads_started":1}
W20260927 10:49:16.939699 18568 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:16.934108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:50921 (local address 127.9.39.190:43935)
0927 10:49:16.934264 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:16.934269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:16.934401 (+ 132us) server_negotiation.cc:407] Connection header received
0927 10:49:16.934544 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:16.934548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:16.934600 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:16.934682 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:16.935139 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.935652 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.936480 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:16.936701 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:16.936821 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:16.939573 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:50921: BlockingRecv error: recv got EOF from 127.9.39.131:50921 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 27 10:49:17 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:17 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:17.582101 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:17.577189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:44571)
0927 10:49:17.577441 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:17.577455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:17.577546 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:17.577750 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:17.577752 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:17.577939 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:17.578027 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:17.578032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.578731 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.578733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:17.579308 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:17.579315 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.579703 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.579705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:17.579765 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:17.580248 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:17.580259 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:17.581996 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
W20260927 10:49:17.582261 18573 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:17.577222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:44571 (local address 127.9.39.190:43935)
0927 10:49:17.577538 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:17.577542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:17.577554 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:49:17.577608 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:17.577610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:17.577648 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:17.577717 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:17.578124 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.578627 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.579397 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.579613 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.579688 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:17.582122 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:44571: BlockingRecv error: recv got EOF from 127.9.39.129:44571 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":68,"threads_started":1}
Sep 27 10:49:17 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:17 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:17.754186 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:17.747966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:37235)
0927 10:49:17.748228 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:17.748243 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:17.748343 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:17.748614 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:17.748617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:17.748813 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:17.748949 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:17.748956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.749786 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.749791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:17.750529 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:17.750540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.751211 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.751214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:17.751305 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:17.751917 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:17.751933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:17.754071 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":102,"threads_started":1}
W20260927 10:49:17.754447 18573 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:17.748138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:37235 (local address 127.9.39.190:43935)
0927 10:49:17.748280 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:17.748286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:17.748302 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:49:17.748459 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:17.748464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:17.748512 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:17.748595 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:17.749060 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.749683 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.750655 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.751078 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.751196 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:17.754201 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:37235: BlockingRecv error: recv got EOF from 127.9.39.130:37235 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 10:49:17 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:17 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:17.945647 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:17.940351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:34229)
0927 10:49:17.940641 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:17.940657 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:17.940763 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:17.941041 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:17.941045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:17.941287 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:17.941430 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:17.941436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.942120 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.942122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:17.942693 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:17.942699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.943138 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.943140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:17.943200 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:17.943720 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:17.943731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:17.945522 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
W20260927 10:49:17.945770 18573 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:17.940475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34229 (local address 127.9.39.190:43935)
0927 10:49:17.940632 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:17.940637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:17.940725 (+ 88us) server_negotiation.cc:407] Connection header received
0927 10:49:17.940865 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:17.940868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:17.940927 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:17.941007 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:17.941534 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.942037 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.942809 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:17.943044 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:17.943120 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:17.945654 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:34229: BlockingRecv error: recv got EOF from 127.9.39.131:34229 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 10:49:18 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:18 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:18.588482 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:18.582875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:60101)
0927 10:49:18.583162 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:18.583176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:18.583263 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:18.583518 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:18.583520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:18.583694 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:18.583788 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:18.583793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.584547 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.584550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:18.585141 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:18.585150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.585633 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.585638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:18.585729 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:18.586352 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:18.586366 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:18.588355 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":83,"threads_started":1}
W20260927 10:49:18.588617 18577 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:18.582988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:60101 (local address 127.9.39.190:43935)
0927 10:49:18.583217 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:18.583220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:18.583233 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:49:18.583359 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:18.583361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:18.583405 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:18.583477 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:18.583907 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.584437 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.585241 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.585470 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.585555 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:18.588491 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:60101: BlockingRecv error: recv got EOF from 127.9.39.129:60101 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":55,"threads_started":1}
Sep 27 10:49:18 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:18 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:18.760897 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:18.754993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:45047)
0927 10:49:18.755257 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:18.755273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:18.755390 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:18.755668 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:18.755671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:18.756013 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:18.756122 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:18.756128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.757208 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.757212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:18.757917 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:18.757926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.758402 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.758405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:18.758488 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:18.758992 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:18.759003 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:18.760707 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":132,"threads_started":1}
W20260927 10:49:18.760910 18577 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:18.755059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:45047 (local address 127.9.39.190:43935)
0927 10:49:18.755196 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:18.755200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:18.755356 (+ 156us) server_negotiation.cc:407] Connection header received
0927 10:49:18.755499 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:18.755505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:18.755557 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:18.755651 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:18.756229 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.757118 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.758024 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.758278 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.758396 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:18.760814 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:45047: BlockingRecv error: recv got EOF from 127.9.39.130:45047 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 10:49:18 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:18 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:18.951617 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:18.946429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:48141)
0927 10:49:18.946717 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:18.946730 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:18.946828 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:18.947036 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:18.947038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:18.947208 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:18.947302 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:18.947307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.947980 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.947982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:18.948518 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:18.948524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.948913 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.948915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:18.948975 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:18.949445 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:18.949459 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:18.951501 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
W20260927 10:49:18.951759 18577 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:18.946476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:48141 (local address 127.9.39.190:43935)
0927 10:49:18.946610 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:18.946613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:18.946796 (+ 183us) server_negotiation.cc:407] Connection header received
0927 10:49:18.946892 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:18.946894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:18.946936 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:18.947004 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:18.947392 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.947893 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.948613 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:18.948826 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:18.948900 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:18.951616 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:48141: BlockingRecv error: recv got EOF from 127.9.39.131:48141 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 10:49:19 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:19 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:19.594154 18580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:19.589190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:40569)
0927 10:49:19.589456 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:19.589469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:19.589551 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:19.589783 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:19.589785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:19.589957 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:19.590049 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:19.590054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.590773 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.590775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:19.591312 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:19.591318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.591714 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.591717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:19.591778 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:19.592308 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:19.592320 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:19.594043 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
W20260927 10:49:19.594291 18581 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:19.589278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:40569 (local address 127.9.39.190:43935)
0927 10:49:19.589495 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:19.589499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:19.589511 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:49:19.589627 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:19.589629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:19.589670 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:19.589745 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:19.590143 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.590676 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.591412 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.591627 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.591700 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:19.594165 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:40569: BlockingRecv error: recv got EOF from 127.9.39.129:40569 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
Sep 27 10:49:19 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:19 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:19.767248 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:19.761826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:54871)
0927 10:49:19.762075 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:19.762088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:19.762189 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:19.762396 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:19.762398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:19.762592 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:19.762684 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:19.762689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.763427 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.763430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:19.763989 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:19.763996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.764381 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.764384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:19.764441 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:19.764919 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:19.764933 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:19.767083 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
W20260927 10:49:19.767292 18581 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:19.761878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:54871 (local address 127.9.39.190:43935)
0927 10:49:19.762014 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:19.762019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:19.762146 (+ 127us) server_negotiation.cc:407] Connection header received
0927 10:49:19.762254 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:19.762256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:19.762297 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:19.762368 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:19.762820 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.763336 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.764078 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.764284 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.764367 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:19.767194 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:54871: BlockingRecv error: recv got EOF from 127.9.39.130:54871 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 27 10:49:19 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:19 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:19.957248 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:19.952329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:44867)
0927 10:49:19.952595 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:19.952608 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:19.952702 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:19.952912 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:19.952914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:19.953131 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:19.953240 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:19.953246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.953952 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.953955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:19.954534 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:19.954542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.954962 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.954964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:19.955022 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:19.955478 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:19.955488 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:19.957139 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":104,"threads_started":1}
W20260927 10:49:19.957384 18581 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:19.952403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:44867 (local address 127.9.39.190:43935)
0927 10:49:19.952533 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:19.952537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:19.952671 (+ 134us) server_negotiation.cc:407] Connection header received
0927 10:49:19.952779 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:19.952781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:19.952825 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:19.952901 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:19.953333 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.953856 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.954629 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:19.954871 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:19.954951 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:19.957259 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:44867: BlockingRecv error: recv got EOF from 127.9.39.131:44867 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 27 10:49:20 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:20 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:20.601200 18584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:20.594980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:41073)
0927 10:49:20.595302 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:20.595318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:20.595449 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:20.595709 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:20.595711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:20.595933 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:20.596059 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:20.596069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.596810 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.596815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:20.597528 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:20.597537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.598036 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.598039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:20.598122 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:20.598677 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:20.598690 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:20.601050 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":70,"threads_started":1}
W20260927 10:49:20.601397 18585 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:20.595017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:41073 (local address 127.9.39.190:43935)
0927 10:49:20.595268 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:20.595275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:20.595394 (+ 119us) server_negotiation.cc:407] Connection header received
0927 10:49:20.595546 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:20.595548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:20.595599 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:20.595690 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:20.596174 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.596702 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.597635 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.597930 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.598035 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:20.601230 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:41073: BlockingRecv error: recv got EOF from 127.9.39.129:41073 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
Sep 27 10:49:20 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:20 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:20.773758 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:20.768069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:35237)
0927 10:49:20.768370 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:20.768387 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:20.768501 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:20.768776 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:20.768779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:20.769010 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:20.769102 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:20.769107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.769881 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.769886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:20.770511 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:20.770520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.771056 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.771059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:20.771148 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:20.771613 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:20.771624 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:20.773542 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
W20260927 10:49:20.773829 18585 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:20.768232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:35237 (local address 127.9.39.190:43935)
0927 10:49:20.768376 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:20.768383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:20.768442 (+ 59us) server_negotiation.cc:407] Connection header received
0927 10:49:20.768611 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:20.768614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:20.768683 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:20.768756 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:20.769212 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.769786 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.770626 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.770948 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.771053 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:20.773686 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:35237: BlockingRecv error: recv got EOF from 127.9.39.130:35237 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 27 10:49:20 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:20 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:20.963603 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:20.958071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:45917)
0927 10:49:20.958349 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:20.958374 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:20.958474 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:20.958732 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:20.958735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:20.958927 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:20.959018 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:20.959023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.959864 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.959872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:20.960600 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:20.960609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.961057 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.961060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:20.961144 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:20.961632 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:20.961643 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:20.963481 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":106,"threads_started":1}
W20260927 10:49:20.963747 18585 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:20.958151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:45917 (local address 127.9.39.190:43935)
0927 10:49:20.958309 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:20.958313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:20.958443 (+ 130us) server_negotiation.cc:407] Connection header received
0927 10:49:20.958547 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:20.958549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:20.958593 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:20.958662 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:20.959112 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.959736 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.960715 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:20.960944 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:20.961050 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:20.963601 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:45917: BlockingRecv error: recv got EOF from 127.9.39.131:45917 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 27 10:49:21 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:21 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:21.610867 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:21.602021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:48537)
0927 10:49:21.602294 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:21.602312 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:21.602421 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:21.602645 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:21.602647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:21.602839 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:21.602930 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:21.602935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.603615 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.603617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:21.604149 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:21.604155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.604525 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.604527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:21.604584 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:21.608691 (+ 4107us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:21.608704 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:21.610745 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
W20260927 10:49:21.611035 18589 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:21.602103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:48537 (local address 127.9.39.190:43935)
0927 10:49:21.602411 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:21.602415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:21.602427 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:49:21.602497 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:21.602500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:21.602539 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:21.602615 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:21.603018 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.603532 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.604234 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.604441 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.604510 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:21.610875 (+ 6365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:48537: BlockingRecv error: recv got EOF from 127.9.39.129:48537 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":68,"threads_started":1}
Sep 27 10:49:21 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:21 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:21.779955 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:21.774603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:50433)
0927 10:49:21.774868 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:21.774883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:21.774995 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:21.775214 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:21.775215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:21.775395 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:21.775483 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:21.775488 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.776188 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.776190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:21.776755 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:21.776762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.777149 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.777152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:21.777212 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:21.777729 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:21.777740 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:21.779749 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":112,"threads_started":1}
W20260927 10:49:21.780011 18589 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:21.774636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:50433 (local address 127.9.39.190:43935)
0927 10:49:21.774792 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:21.774796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:21.774972 (+ 176us) server_negotiation.cc:407] Connection header received
0927 10:49:21.775071 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:21.775073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:21.775120 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:21.775186 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:21.775567 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.776098 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.776846 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.777065 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.777133 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:21.779903 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:50433: BlockingRecv error: recv got EOF from 127.9.39.130:50433 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 27 10:49:21 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:21 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.131@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:21.969749 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:21.964302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.131:34579)
0927 10:49:21.964562 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:21.964576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:21.964679 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:21.964926 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:21.964929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:21.965105 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:21.965194 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:21.965199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.965904 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.965906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:21.966474 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:21.966481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.966898 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.966900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:21.966962 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:21.967474 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:21.967486 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:21.969577 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
W20260927 10:49:21.969883 18589 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:21.964424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:34579 (local address 127.9.39.190:43935)
0927 10:49:21.964569 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:21.964572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:21.964645 (+ 73us) server_negotiation.cc:407] Connection header received
0927 10:49:21.964756 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:21.964758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:21.964825 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:21.964889 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:21.965287 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.965818 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.966565 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:21.966812 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:21.966886 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:21.969716 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.131:34579: BlockingRecv error: recv got EOF from 127.9.39.131:34579 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 27 10:49:22 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:22 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.129@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:22.616554 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:22.611719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.129:54279)
0927 10:49:22.611971 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:22.611985 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:22.612068 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:22.612310 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:22.612312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:22.612481 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:22.612566 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:22.612571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.613231 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.613233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:22.613819 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:22.613827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.614230 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.614232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:22.614290 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:22.614765 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:22.614790 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:22.616400 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
W20260927 10:49:22.616637 18593 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:22.611811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:54279 (local address 127.9.39.190:43935)
0927 10:49:22.612035 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:22.612039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:22.612051 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:49:22.612151 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:22.612153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:22.612195 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:22.612278 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:22.612653 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.613149 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.613919 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.614146 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.614219 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:22.616515 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.129:54279: BlockingRecv error: recv got EOF from 127.9.39.129:54279 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":57,"threads_started":1}
Sep 27 10:49:22 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 10:49:22 dist-test-slave-p9q6 krb5kdc[17535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.39.130@KRBTEST.COM for oryx/127.9.39.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 10:49:22.785738 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:22.780852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:43935 (local address 127.9.39.130:54825)
0927 10:49:22.781103 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:22.781115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:22.781203 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:22.781401 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:22.781403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:22.781568 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:22.781653 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:22.781658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.782379 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.782381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:22.783011 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:22.783020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.783435 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.783437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:22.783494 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:22.783932 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:22.783942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:22.785622 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.39.190:43935: Server oryx/127.9.39.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":99,"threads_started":1}
W20260927 10:49:22.785921 18593 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:22.780893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:54825 (local address 127.9.39.190:43935)
0927 10:49:22.781037 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:22.781042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:22.781175 (+ 133us) server_negotiation.cc:407] Connection header received
0927 10:49:22.781265 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:22.781267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:22.781309 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:22.781374 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:22.781757 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.782291 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.783109 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:22.783335 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:22.783423 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:22.785789 (+ 2366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.39.130:54825: BlockingRecv error: recv got EOF from 127.9.39.130:54825 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:49:22.865128 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18118
I20260927 10:49:22.870358 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18270
I20260927 10:49:22.875337 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18402
I20260927 10:49:22.880388 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18048
2026-09-27T10:49:22Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17969 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T10:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:22Z Disabled control of system clock
I20260927 10:49:22.909439 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42001
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:44887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:42001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 10:49:23.017807 18598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:23.018069 18598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:23.018101 18598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:23.022033 18598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:23.022096 18598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:23.022111 18598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:23.022127 18598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:23.022142 18598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:23.026933 18598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:42001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:23.027877 18598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:23.029172 18598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:23.037307 18604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.037384 18603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.037421 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:23.037464 18598 server_base.cc:1034] running on GCE node
I20260927 10:49:23.038162 18598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:23.039170 18598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:23.040356 18598 hybrid_clock.cc:648] HybridClock initialized: now 1790506163040326 us; error 43 us; skew 500 ppm
I20260927 10:49:23.042060 18598 webserver.cc:460] Webserver started at http://127.9.39.190:40943/ using document root <none> and password file <none>
I20260927 10:49:23.042594 18598 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:23.042652 18598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:23.042920 18598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:23.044886 18598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "6e7a3f915d9b4808b75c6e817e3e7402"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "b8872c6e23e005707a6557882c05562a"
server_key_iv: "128a206aad64d185bd6848b52fdbf72b"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.045439 18598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "6e7a3f915d9b4808b75c6e817e3e7402"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "b8872c6e23e005707a6557882c05562a"
server_key_iv: "128a206aad64d185bd6848b52fdbf72b"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.048890 18598 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260927 10:49:23.051218 18612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.052392 18598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:49:23.052517 18598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "6e7a3f915d9b4808b75c6e817e3e7402"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "b8872c6e23e005707a6557882c05562a"
server_key_iv: "128a206aad64d185bd6848b52fdbf72b"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.052618 18598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:23.070340 18598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:23.071054 18598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:23.071241 18598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:23.077881 18598 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:42001
I20260927 10:49:23.077898 18664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:42001 every 8 connection(s)
I20260927 10:49:23.078795 18598 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:49:23.081463 18665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:23.084690 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18598
I20260927 10:49:23.084817 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:49:23.085093 9374 external_mini_cluster.cc:1428] Setting key 92ad064409ca2f5a504f7da2062f7c00
I20260927 10:49:23.087314 18665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402: Bootstrap starting.
I20260927 10:49:23.089615 18665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:23.090344 18665 log.cc:824] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:23.092240 18665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402: No bootstrap required, opened a new log
I20260927 10:49:23.093659 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.086485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.9.39.190:42001)
0927 10:49:23.087115 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:23.087130 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:23.087164 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:49:23.087785 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:23.087801 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:23.088166 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:23.088433 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:23.088778 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.089705 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.090601 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.091120 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.091296 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.092107 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:23.092112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:23.092123 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:23.092484 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:23.092536 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:23.092554 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:23.092654 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:23.092788 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 10:49:23.093020 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":136,"threads_started":1}
I20260927 10:49:23.095340 18665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42001 } }
I20260927 10:49:23.095566 18665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:23.095625 18665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7a3f915d9b4808b75c6e817e3e7402, State: Initialized, Role: FOLLOWER
I20260927 10:49:23.096091 18665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42001 } }
I20260927 10:49:23.096246 18665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:49:23.096310 18665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:49:23.096403 18665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:23.097216 18665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42001 } }
I20260927 10:49:23.097530 18665 leader_election.cc:302] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e7a3f915d9b4808b75c6e817e3e7402; no voters:
I20260927 10:49:23.097800 18665 leader_election.cc:288] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:49:23.097957 18670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:23.098194 18670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [term 1 LEADER]: Becoming Leader. State: Replica: 6e7a3f915d9b4808b75c6e817e3e7402, State: Running, Role: LEADER
I20260927 10:49:23.098520 18670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42001 } }
I20260927 10:49:23.098968 18665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:49:23.100342 18671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42001 } } }
I20260927 10:49:23.100473 18671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:23.100788 18672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e7a3f915d9b4808b75c6e817e3e7402. Latest consensus state: current_term: 1 leader_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7a3f915d9b4808b75c6e817e3e7402" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42001 } } }
I20260927 10:49:23.100924 18681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:49:23.101827 18672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:23.103763 18681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:49:23.109228 18681 catalog_manager.cc:1347] Generated new cluster ID: d27a884e3de446b3a16583c8c288c831
I20260927 10:49:23.109290 18681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:49:23.115458 18681 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:49:23.116492 18681 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:49:23.122112 18681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e7a3f915d9b4808b75c6e817e3e7402: Generated new TSK 0
I20260927 10:49:23.122686 18681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:49:23.125489 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42001
--builtin_ntp_servers=127.9.39.148:44887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 10:49:23.231770 18689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:23.231994 18689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:23.232029 18689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:23.236229 18689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:23.236284 18689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:23.236335 18689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:49:23.241125 18689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:23.242108 18689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:23.243461 18689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:23.252724 18695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.252732 18697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.252709 18694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:23.252987 18689 server_base.cc:1034] running on GCE node
I20260927 10:49:23.253509 18689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:23.254089 18689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:23.255268 18689 hybrid_clock.cc:648] HybridClock initialized: now 1790506163255256 us; error 30 us; skew 500 ppm
I20260927 10:49:23.257114 18689 webserver.cc:460] Webserver started at http://127.9.39.129:34443/ using document root <none> and password file <none>
I20260927 10:49:23.257653 18689 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:23.257736 18689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:23.257949 18689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:23.259899 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "96a3cabfc3c744bb8ff24f431221d1da"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "aec6ea6440956d53b50433096c53edde"
server_key_iv: "faadcd1708593b178ab29f5f1b74343c"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.260438 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "96a3cabfc3c744bb8ff24f431221d1da"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "aec6ea6440956d53b50433096c53edde"
server_key_iv: "faadcd1708593b178ab29f5f1b74343c"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.263762 18689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:49:23.265990 18703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.267158 18689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:23.267284 18689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "96a3cabfc3c744bb8ff24f431221d1da"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "aec6ea6440956d53b50433096c53edde"
server_key_iv: "faadcd1708593b178ab29f5f1b74343c"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.267383 18689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:23.278592 18689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:23.279218 18689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:23.279402 18689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:23.279917 18689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:23.280738 18689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:23.280815 18689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.280884 18689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:23.280926 18689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.288870 18689 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:41187
I20260927 10:49:23.288893 18816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:41187 every 8 connection(s)
I20260927 10:49:23.289677 18689 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:49:23.290957 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18689
I20260927 10:49:23.291064 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:49:23.291370 9374 external_mini_cluster.cc:1428] Setting key 84ecc04e6abf47799f2e19234679c7f4
I20260927 10:49:23.293408 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42001
--builtin_ntp_servers=127.9.39.148:44887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 10:49:23.299160 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.291384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:55767 (local address 127.9.39.190:42001)
0927 10:49:23.291520 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:23.291525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:23.292600 (+ 1075us) server_negotiation.cc:407] Connection header received
0927 10:49:23.294526 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:23.294529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:23.294572 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:23.294634 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:23.295582 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.296113 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.297234 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.297455 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.297556 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.298621 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:23.298629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:23.298631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:23.298642 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:23.298660 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:23.298667 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:23.298765 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:23.298962 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 10:49:23.299019 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:49:23.300012 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.291817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42001 (local address 127.9.39.129:55767)
0927 10:49:23.292452 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:23.292489 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:23.294276 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:23.294860 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:23.294873 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:23.294991 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:23.295454 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:23.295466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.296231 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.296234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:23.297122 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:23.297131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.297587 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.297591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:23.297739 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.298385 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:49:23.298387 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:23.298422 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:49:23.298480 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:23.298784 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:23.298799 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:23.298804 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:49:23.298947 (+ 143us) client_negotiation.cc:240] Negotiation successful
0927 10:49:23.299256 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":110,"threads_started":1}
I20260927 10:49:23.301188 18817 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42001
I20260927 10:49:23.301443 18817 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:23.302088 18817 heartbeater.cc:511] Master 127.9.39.190:42001 requested a full tablet report, sending...
I20260927 10:49:23.303618 18629 ts_manager.cc:194] Registered new tserver with Master: 96a3cabfc3c744bb8ff24f431221d1da (127.9.39.129:41187)
I20260927 10:49:23.304711 18629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.129:55767
W20260927 10:49:23.400354 18820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:23.400604 18820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:23.400676 18820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:23.404691 18820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:23.404747 18820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:23.404796 18820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:49:23.409498 18820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:23.410437 18820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:23.411813 18820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:23.420521 18825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.420527 18828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.420607 18826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:23.420884 18820 server_base.cc:1034] running on GCE node
I20260927 10:49:23.421237 18820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:23.421819 18820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:23.422969 18820 hybrid_clock.cc:648] HybridClock initialized: now 1790506163422950 us; error 31 us; skew 500 ppm
I20260927 10:49:23.424654 18820 webserver.cc:460] Webserver started at http://127.9.39.130:44407/ using document root <none> and password file <none>
I20260927 10:49:23.425216 18820 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:23.425278 18820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:23.425522 18820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:23.427462 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "a5855464e5ce4ed9aca59ad6eb3d7b53"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "e2d034f3602bc62ce3f75b5c98f5d900"
server_key_iv: "9330decfebe4f5497491345c61f0e0d8"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.427966 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "a5855464e5ce4ed9aca59ad6eb3d7b53"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "e2d034f3602bc62ce3f75b5c98f5d900"
server_key_iv: "9330decfebe4f5497491345c61f0e0d8"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.431324 18820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:49:23.433535 18834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.434752 18820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:23.434856 18820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "a5855464e5ce4ed9aca59ad6eb3d7b53"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "e2d034f3602bc62ce3f75b5c98f5d900"
server_key_iv: "9330decfebe4f5497491345c61f0e0d8"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.434967 18820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:23.452310 18820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:23.452857 18820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:23.452988 18820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:23.453452 18820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:23.454269 18820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:23.454313 18820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.454349 18820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:23.454371 18820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.462746 18820 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:43429
I20260927 10:49:23.462785 18947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:43429 every 8 connection(s)
I20260927 10:49:23.463588 18820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:49:23.469213 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18820
I20260927 10:49:23.469439 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:49:23.469772 9374 external_mini_cluster.cc:1428] Setting key c8fa1ed94a01ec06c9dd7176b2dff32a
I20260927 10:49:23.471922 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42001
--builtin_ntp_servers=127.9.39.148:44887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 10:49:23.473207 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.465368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:56959 (local address 127.9.39.190:42001)
0927 10:49:23.465507 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:23.465510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:23.466582 (+ 1072us) server_negotiation.cc:407] Connection header received
0927 10:49:23.467937 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:23.467940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:23.467987 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:23.468064 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:23.468903 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.469493 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.471001 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.471318 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.471401 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.472705 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:23.472709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:23.472711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:23.472724 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:23.472760 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:23.472769 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:23.472843 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:23.473010 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 10:49:23.473071 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 10:49:23.474087 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.465789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42001 (local address 127.9.39.130:56959)
0927 10:49:23.466423 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:23.466461 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:23.467714 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:23.468218 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:23.468224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:23.468304 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:23.468761 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:23.468772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.469623 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.469627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:23.470886 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:23.470897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.471463 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.471467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:23.471672 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.472414 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:49:23.472418 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:23.472473 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:49:23.472558 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:23.472856 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:23.472866 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:23.472872 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 10:49:23.472999 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 10:49:23.473293 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
I20260927 10:49:23.475242 18948 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42001
I20260927 10:49:23.475462 18948 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:23.475950 18948 heartbeater.cc:511] Master 127.9.39.190:42001 requested a full tablet report, sending...
I20260927 10:49:23.476953 18629 ts_manager.cc:194] Registered new tserver with Master: a5855464e5ce4ed9aca59ad6eb3d7b53 (127.9.39.130:43429)
I20260927 10:49:23.477511 18629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.130:56959
W20260927 10:49:23.578990 18951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:23.579226 18951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:23.579282 18951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:23.583195 18951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:23.583238 18951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:23.583323 18951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:49:23.587984 18951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:23.588963 18951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:23.590281 18951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:23.599481 18956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.599491 18957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.599475 18959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:23.599767 18951 server_base.cc:1034] running on GCE node
I20260927 10:49:23.600376 18951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:23.600980 18951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:23.602151 18951 hybrid_clock.cc:648] HybridClock initialized: now 1790506163602129 us; error 38 us; skew 500 ppm
I20260927 10:49:23.603899 18951 webserver.cc:460] Webserver started at http://127.9.39.131:35495/ using document root <none> and password file <none>
I20260927 10:49:23.604466 18951 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:23.604553 18951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:23.604763 18951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:23.606679 18951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "5808f19b43d24f4c97a60e53c7c2fc4c"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "a8409c23b391dd941f08c9b0f25cef60"
server_key_iv: "3f2217d2d3c115e5e2bc5cf70bce60d2"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.607271 18951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "5808f19b43d24f4c97a60e53c7c2fc4c"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "a8409c23b391dd941f08c9b0f25cef60"
server_key_iv: "3f2217d2d3c115e5e2bc5cf70bce60d2"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.611091 18951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260927 10:49:23.613377 18965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.614493 18951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:23.614610 18951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "5808f19b43d24f4c97a60e53c7c2fc4c"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "a8409c23b391dd941f08c9b0f25cef60"
server_key_iv: "3f2217d2d3c115e5e2bc5cf70bce60d2"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.614737 18951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:23.630018 18951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:23.630617 18951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:23.630823 18951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:23.631340 18951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:23.632172 18951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:23.632236 18951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.632318 18951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:23.632361 18951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.641052 18951 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:35203
I20260927 10:49:23.641073 19078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:35203 every 8 connection(s)
I20260927 10:49:23.641911 18951 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:49:23.648200 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 18951
I20260927 10:49:23.648331 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:49:23.648576 9374 external_mini_cluster.cc:1428] Setting key 826ab60999bbf7be3522e39ad876c54a
I20260927 10:49:23.651027 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.643678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:48243 (local address 127.9.39.190:42001)
0927 10:49:23.643829 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:23.643834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:23.644785 (+ 951us) server_negotiation.cc:407] Connection header received
0927 10:49:23.646067 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:23.646069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:23.646111 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:23.646186 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:23.647190 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.647692 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.649018 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.649340 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.649450 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.650569 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:23.650577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:23.650579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:23.650590 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:23.650623 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:23.650632 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:23.650733 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:23.650882 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 10:49:23.650926 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 10:49:23.651845 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.644061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42001 (local address 127.9.39.131:48243)
0927 10:49:23.644645 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:23.644680 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:23.645894 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:23.646375 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:23.646382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:23.646470 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:23.647069 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:23.647082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.647829 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.647831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:23.648882 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:23.648893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:23.649494 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:23.649498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:23.649670 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:23.650289 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:49:23.650291 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:23.650345 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:49:23.650421 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:23.650748 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:23.650757 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:23.650762 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:49:23.650872 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 10:49:23.651093 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260927 10:49:23.652860 19079 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42001
I20260927 10:49:23.653072 19079 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:23.653563 19079 heartbeater.cc:511] Master 127.9.39.190:42001 requested a full tablet report, sending...
I20260927 10:49:23.654480 18629 ts_manager.cc:194] Registered new tserver with Master: 5808f19b43d24f4c97a60e53c7c2fc4c (127.9.39.131:35203)
I20260927 10:49:23.655114 18629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.131:48243
I20260927 10:49:23.662106 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 10:49:23.664671 18668 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:23.663924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49074 (local address 127.9.39.190:42001)
0927 10:49:23.664032 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:23.664036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:23.664164 (+ 128us) server_negotiation.cc:407] Connection header received
0927 10:49:23.664260 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:23.664262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:23.664296 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:23.664359 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:23.664539 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49074: BlockingRecv error: recv got EOF from 127.0.0.1:49074 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 10:49:23.665023 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18689
I20260927 10:49:23.670449 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18820
I20260927 10:49:23.676179 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18951
I20260927 10:49:23.681331 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 18598
2026-09-27T10:49:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (800 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T10:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:23Z Disabled control of system clock
I20260927 10:49:23.709667 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45363
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 10:49:23.814826 19090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:23.815058 19090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:23.815093 19090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:23.818831 19090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:23.818874 19090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:23.818889 19090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:23.818904 19090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:23.818919 19090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:23.823374 19090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:45363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:23.824316 19090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:23.825546 19090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:23.833477 19095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:23.833900 19098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:23.833972 19090 server_base.cc:1034] running on GCE node
W20260927 10:49:23.834044 19096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:23.834555 19090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:23.835633 19090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:23.836804 19090 hybrid_clock.cc:648] HybridClock initialized: now 1790506163836774 us; error 40 us; skew 500 ppm
I20260927 10:49:23.838511 19090 webserver.cc:460] Webserver started at http://127.9.39.190:42663/ using document root <none> and password file <none>
I20260927 10:49:23.839155 19090 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:23.839216 19090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:23.839427 19090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:23.841323 19090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "fb043f3ebe534ad5b346415574099b74"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "c95bc5021b4e18e88945d6662c8491b7"
server_key_iv: "8fe3bd937abcc08c92a1e42cfced197a"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.841836 19090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "fb043f3ebe534ad5b346415574099b74"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "c95bc5021b4e18e88945d6662c8491b7"
server_key_iv: "8fe3bd937abcc08c92a1e42cfced197a"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.845144 19090 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260927 10:49:23.847420 19104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:23.848475 19090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:49:23.848599 19090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "fb043f3ebe534ad5b346415574099b74"
format_stamp: "Formatted at 2026-09-27 10:49:23 on dist-test-slave-p9q6"
server_key: "c95bc5021b4e18e88945d6662c8491b7"
server_key_iv: "8fe3bd937abcc08c92a1e42cfced197a"
server_key_version: "encryptionkey@0"
I20260927 10:49:23.848718 19090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:23.852960 19090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:23.853492 19090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:23.853652 19090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:23.860093 19090 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:45363
I20260927 10:49:23.860109 19156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:45363 every 8 connection(s)
I20260927 10:49:23.860908 19090 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:49:23.863955 19157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:23.864568 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19090
I20260927 10:49:23.864697 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:49:23.864988 9374 external_mini_cluster.cc:1428] Setting key e371ef28316432c2a36ffc4c06aebb9d
I20260927 10:49:23.869984 19157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74: Bootstrap starting.
I20260927 10:49:23.871062 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:23.866337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.9.39.190:45363)
0927 10:49:23.866962 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:23.866976 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:23.867008 (+ 32us) server_negotiation.cc:407] Connection header received
0927 10:49:23.867620 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:23.867634 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:23.867961 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:23.868319 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:23.869579 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:23.869585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:23.869600 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:23.869612 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:23.869716 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:23.869739 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:23.869884 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:23.869983 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 10:49:23.870216 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260927 10:49:23.872447 19157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:23.873279 19157 log.cc:824] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:23.875296 19157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74: No bootstrap required, opened a new log
I20260927 10:49:23.878934 19157 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb043f3ebe534ad5b346415574099b74" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45363 } }
I20260927 10:49:23.879132 19157 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:23.879209 19157 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb043f3ebe534ad5b346415574099b74, State: Initialized, Role: FOLLOWER
I20260927 10:49:23.879650 19157 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb043f3ebe534ad5b346415574099b74" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45363 } }
I20260927 10:49:23.879770 19157 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:49:23.879845 19157 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:49:23.879935 19157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:23.880834 19157 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb043f3ebe534ad5b346415574099b74" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45363 } }
I20260927 10:49:23.881140 19157 leader_election.cc:302] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb043f3ebe534ad5b346415574099b74; no voters:
I20260927 10:49:23.881412 19157 leader_election.cc:288] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:49:23.881526 19162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:23.881778 19162 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [term 1 LEADER]: Becoming Leader. State: Replica: fb043f3ebe534ad5b346415574099b74, State: Running, Role: LEADER
I20260927 10:49:23.882139 19162 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb043f3ebe534ad5b346415574099b74" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45363 } }
I20260927 10:49:23.882653 19157 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:49:23.883945 19163 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb043f3ebe534ad5b346415574099b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb043f3ebe534ad5b346415574099b74" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45363 } } }
I20260927 10:49:23.884526 19163 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:23.884078 19164 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb043f3ebe534ad5b346415574099b74. Latest consensus state: current_term: 1 leader_uuid: "fb043f3ebe534ad5b346415574099b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb043f3ebe534ad5b346415574099b74" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45363 } } }
I20260927 10:49:23.884824 19164 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74 [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:23.884835 19176 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 10:49:23.885190 19177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:49:23.887866 19177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:49:23.892989 19177 catalog_manager.cc:1347] Generated new cluster ID: b9978ea994cf40b9bba294031d2cc70f
I20260927 10:49:23.893042 19177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:49:23.906065 19177 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:49:23.906344 19177 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:49:23.910427 19177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb043f3ebe534ad5b346415574099b74: Generated new TSK 0
I20260927 10:49:23.911036 19177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:49:23.915217 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45363
--builtin_ntp_servers=127.9.39.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 10:49:24.022169 19181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:24.022457 19181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:24.022535 19181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:24.026578 19181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:24.026669 19181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:24.026799 19181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:49:24.031725 19181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:24.032757 19181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:24.034078 19181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:24.043473 19187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:24.043637 19181 server_base.cc:1034] running on GCE node
W20260927 10:49:24.043499 19189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:24.043501 19186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:24.044119 19181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:24.044695 19181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:24.045866 19181 hybrid_clock.cc:648] HybridClock initialized: now 1790506164045849 us; error 28 us; skew 500 ppm
I20260927 10:49:24.047655 19181 webserver.cc:460] Webserver started at http://127.9.39.129:38069/ using document root <none> and password file <none>
I20260927 10:49:24.048184 19181 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:24.048267 19181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:24.048527 19181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:24.050460 19181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "1a30318ea6c644fa9ee34685003cb704"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "25a028c649e96fd229dd44631d5be0df"
server_key_iv: "806f774c6b6ed3b2f916c5e09b7281d8"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.051048 19181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "1a30318ea6c644fa9ee34685003cb704"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "25a028c649e96fd229dd44631d5be0df"
server_key_iv: "806f774c6b6ed3b2f916c5e09b7281d8"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.054680 19181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:49:24.057124 19195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.058256 19181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 10:49:24.058400 19181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "1a30318ea6c644fa9ee34685003cb704"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "25a028c649e96fd229dd44631d5be0df"
server_key_iv: "806f774c6b6ed3b2f916c5e09b7281d8"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.058507 19181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:24.062978 19181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:24.063544 19181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:24.063714 19181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:24.064213 19181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:24.065042 19181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:24.065109 19181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.065167 19181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:24.065210 19181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.073400 19181 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:43779
I20260927 10:49:24.073434 19308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:43779 every 8 connection(s)
I20260927 10:49:24.074239 19181 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:49:24.080041 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:24.076018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:38615 (local address 127.9.39.190:45363)
0927 10:49:24.076155 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:24.076159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:24.077199 (+ 1040us) server_negotiation.cc:407] Connection header received
0927 10:49:24.078388 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:24.078391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:24.078427 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:24.078501 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:24.079635 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:24.079643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:24.079645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:24.079657 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:24.079677 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:24.079686 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:24.079780 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:24.079899 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 10:49:24.079937 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:49:24.080971 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:24.076426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45363 (local address 127.9.39.129:38615)
0927 10:49:24.077062 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:24.077100 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:24.078195 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:24.078673 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:24.078680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:24.078818 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:24.079373 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:49:24.079374 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:24.079415 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:49:24.079482 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:24.079781 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:24.079788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:24.079792 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:49:24.079923 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 10:49:24.080136 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":100,"threads_started":1}
I20260927 10:49:24.081125 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19181
I20260927 10:49:24.081236 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:49:24.081581 9374 external_mini_cluster.cc:1428] Setting key 0f8a02ec63c345f803f76e493771caf5
I20260927 10:49:24.082237 19309 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45363
I20260927 10:49:24.082463 19309 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:24.082947 19309 heartbeater.cc:511] Master 127.9.39.190:45363 requested a full tablet report, sending...
I20260927 10:49:24.083562 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45363
--builtin_ntp_servers=127.9.39.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 10:49:24.084627 19121 ts_manager.cc:194] Registered new tserver with Master: 1a30318ea6c644fa9ee34685003cb704 (127.9.39.129:43779)
W20260927 10:49:24.191030 19312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:24.191293 19312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:24.191360 19312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:24.195223 19312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:24.195299 19312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:24.195392 19312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:49:24.199980 19312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:24.201061 19312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:24.202435 19312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:24.211767 19317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:24.211771 19318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:24.211937 19312 server_base.cc:1034] running on GCE node
W20260927 10:49:24.211805 19320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:24.212437 19312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:24.213004 19312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:24.214169 19312 hybrid_clock.cc:648] HybridClock initialized: now 1790506164214144 us; error 44 us; skew 500 ppm
I20260927 10:49:24.215926 19312 webserver.cc:460] Webserver started at http://127.9.39.130:33735/ using document root <none> and password file <none>
I20260927 10:49:24.216491 19312 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:24.216550 19312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:24.216792 19312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:24.218796 19312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "c444a5aa5afd48c196e72fd11453865e"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "c940eb229a61bca83c3f392679c1189d"
server_key_iv: "8217d349082a75a7df53360d6f01c252"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.219395 19312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "c444a5aa5afd48c196e72fd11453865e"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "c940eb229a61bca83c3f392679c1189d"
server_key_iv: "8217d349082a75a7df53360d6f01c252"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.222919 19312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:49:24.225250 19326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.226399 19312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:24.226521 19312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "c444a5aa5afd48c196e72fd11453865e"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "c940eb229a61bca83c3f392679c1189d"
server_key_iv: "8217d349082a75a7df53360d6f01c252"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.226636 19312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:24.231127 19312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:24.231676 19312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:24.231840 19312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:24.232331 19312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:24.233172 19312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:24.233237 19312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.233307 19312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:24.233353 19312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.241597 19312 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:34389
I20260927 10:49:24.241619 19439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:34389 every 8 connection(s)
I20260927 10:49:24.242388 19312 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:49:24.247923 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:24.244041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:37725 (local address 127.9.39.190:45363)
0927 10:49:24.244188 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:24.244193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:24.245144 (+ 951us) server_negotiation.cc:407] Connection header received
0927 10:49:24.246386 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:24.246388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:24.246423 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:24.246484 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:24.247504 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:24.247506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:24.247508 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:24.247519 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:24.247537 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:24.247544 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:24.247618 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:24.247719 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 10:49:24.247754 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:49:24.248649 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:24.244425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45363 (local address 127.9.39.130:37725)
0927 10:49:24.245013 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:24.245048 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:24.246190 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:24.246640 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:24.246646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:24.246807 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:24.247274 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:49:24.247275 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:24.247311 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:49:24.247373 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:24.247612 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:24.247618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:24.247622 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:49:24.247757 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 10:49:24.247943 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":106,"threads_started":1}
I20260927 10:49:24.249675 19440 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45363
I20260927 10:49:24.249938 19440 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:24.249958 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19312
I20260927 10:49:24.250061 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:49:24.250370 9374 external_mini_cluster.cc:1428] Setting key e36ac108b04b96821615130c53eb32b7
I20260927 10:49:24.250372 19440 heartbeater.cc:511] Master 127.9.39.190:45363 requested a full tablet report, sending...
I20260927 10:49:24.251389 19120 ts_manager.cc:194] Registered new tserver with Master: c444a5aa5afd48c196e72fd11453865e (127.9.39.130:34389)
I20260927 10:49:24.252184 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45363
--builtin_ntp_servers=127.9.39.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 10:49:24.364173 19443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:24.364434 19443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:24.364503 19443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:24.368463 19443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:24.368528 19443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:24.368608 19443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:49:24.373435 19443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:24.374526 19443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:24.375883 19443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:24.385190 19451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:24.385164 19448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:24.385320 19449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:24.385610 19443 server_base.cc:1034] running on GCE node
I20260927 10:49:24.385998 19443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:24.386576 19443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:24.387748 19443 hybrid_clock.cc:648] HybridClock initialized: now 1790506164387720 us; error 68 us; skew 500 ppm
I20260927 10:49:24.389539 19443 webserver.cc:460] Webserver started at http://127.9.39.131:33709/ using document root <none> and password file <none>
I20260927 10:49:24.390082 19443 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:24.390169 19443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:24.390391 19443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:24.392359 19443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "874d19a838234137ba837420afd91e0c"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "4e3c458424ae60448bb9c5cc456ff624"
server_key_iv: "bb51345e9c1181983650207eb1117c4d"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.392879 19443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "874d19a838234137ba837420afd91e0c"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "4e3c458424ae60448bb9c5cc456ff624"
server_key_iv: "bb51345e9c1181983650207eb1117c4d"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.396235 19443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:49:24.398617 19457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.399817 19443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:24.399946 19443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "874d19a838234137ba837420afd91e0c"
format_stamp: "Formatted at 2026-09-27 10:49:24 on dist-test-slave-p9q6"
server_key: "4e3c458424ae60448bb9c5cc456ff624"
server_key_iv: "bb51345e9c1181983650207eb1117c4d"
server_key_version: "encryptionkey@0"
I20260927 10:49:24.400051 19443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:24.404301 19443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:24.404836 19443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:24.404997 19443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:24.405510 19443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:24.406347 19443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:24.406414 19443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.406476 19443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:24.406522 19443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:24.415104 19443 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:34761
I20260927 10:49:24.415131 19570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:34761 every 8 connection(s)
I20260927 10:49:24.415885 19443 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:49:24.417815 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19443
I20260927 10:49:24.417934 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:49:24.418223 9374 external_mini_cluster.cc:1428] Setting key 64166fae0e844a6ea193efe66f45dc0e
I20260927 10:49:24.422034 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:24.417597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:36859 (local address 127.9.39.190:45363)
0927 10:49:24.417710 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:24.417714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:24.418998 (+ 1284us) server_negotiation.cc:407] Connection header received
0927 10:49:24.420573 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:24.420575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:24.420609 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:24.420682 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:24.421626 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:24.421629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:24.421631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:49:24.421641 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:24.421658 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:49:24.421692 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:24.421759 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:24.421903 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 10:49:24.421940 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 10:49:24.422848 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:24.418096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45363 (local address 127.9.39.131:36859)
0927 10:49:24.418833 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:24.418883 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:24.420388 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:24.420829 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:24.420835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:24.420915 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:24.421413 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:49:24.421414 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:24.421450 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:49:24.421510 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:24.421759 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:24.421765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:24.421768 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 10:49:24.421897 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 10:49:24.422066 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":122,"threads_started":1}
I20260927 10:49:24.423851 19571 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45363
I20260927 10:49:24.424052 19571 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:24.424420 19571 heartbeater.cc:511] Master 127.9.39.190:45363 requested a full tablet report, sending...
I20260927 10:49:24.425294 19120 ts_manager.cc:194] Registered new tserver with Master: 874d19a838234137ba837420afd91e0c (127.9.39.131:34761)
I20260927 10:49:24.432029 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 10:49:24.434610 19160 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:49:24.433913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.9.39.190:45363)
0927 10:49:24.434016 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:24.434023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:24.434174 (+ 151us) server_negotiation.cc:407] Connection header received
0927 10:49:24.434276 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:24.434278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:24.434305 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:24.434369 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:24.434521 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41394: BlockingRecv error: recv got EOF from 127.0.0.1:41394 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:49:24.434841 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19181
I20260927 10:49:24.440339 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19312
I20260927 10:49:24.445451 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19443
I20260927 10:49:24.450454 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19090
2026-09-27T10:49:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (769 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:49:24 dist-test-slave-p9q6 krb5kdc[19581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:49:24 dist-test-slave-p9q6 krb5kdc[19581](info): set up 2 sockets
Sep 27 10:49:24 dist-test-slave-p9q6 krb5kdc[19581](info): commencing operation
krb5kdc: starting...
W20260927 10:49:25.483834 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:49:25 dist-test-slave-p9q6 krb5kdc[19581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:49:25.606218 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42017
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:25.714977 19594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:25.715255 19594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:25.715296 19594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:25.719265 19594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:49:25.719323 19594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:25.719343 19594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:25.719359 19594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:49:25.719374 19594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:49:25.724550 19594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:42017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:25.725581 19594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:25.726568 19594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:25.735096 19599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:25.735142 19594 server_base.cc:1034] running on GCE node
W20260927 10:49:25.735158 19602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:25.735107 19600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:25.735929 19594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:25.737015 19594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:25.738166 19594 hybrid_clock.cc:648] HybridClock initialized: now 1790506165738139 us; error 42 us; skew 500 ppm
Sep 27 10:49:25 dist-test-slave-p9q6 krb5kdc[19581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:25.740804 19594 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:49:25.741709 19594 webserver.cc:460] Webserver started at http://127.9.39.190:44881/ using document root <none> and password file <none>
I20260927 10:49:25.742259 19594 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:25.742352 19594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:25.742609 19594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:25.744666 19594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "ed69681de1a147e380e72f25dedfef5b"
format_stamp: "Formatted at 2026-09-27 10:49:25 on dist-test-slave-p9q6"
server_key: "b7d23467c7d6d462008a780e47779f1c"
server_key_iv: "8a76b6b6ad2676f5294d0fc1f5586126"
server_key_version: "encryptionkey@0"
I20260927 10:49:25.745163 19594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "ed69681de1a147e380e72f25dedfef5b"
format_stamp: "Formatted at 2026-09-27 10:49:25 on dist-test-slave-p9q6"
server_key: "b7d23467c7d6d462008a780e47779f1c"
server_key_iv: "8a76b6b6ad2676f5294d0fc1f5586126"
server_key_version: "encryptionkey@0"
I20260927 10:49:25.748762 19594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:49:25.751242 19609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:25.752473 19594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:49:25.752609 19594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "ed69681de1a147e380e72f25dedfef5b"
format_stamp: "Formatted at 2026-09-27 10:49:25 on dist-test-slave-p9q6"
server_key: "b7d23467c7d6d462008a780e47779f1c"
server_key_iv: "8a76b6b6ad2676f5294d0fc1f5586126"
server_key_version: "encryptionkey@0"
I20260927 10:49:25.752735 19594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:25.765399 19594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:25.768074 19594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:25.768257 19594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:25.775010 19594 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:42017
I20260927 10:49:25.775282 19661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:42017 every 8 connection(s)
I20260927 10:49:25.775957 19594 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:49:25.778677 19662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:25.782187 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19594
I20260927 10:49:25.782320 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:49:25.782642 9374 external_mini_cluster.cc:1428] Setting key 9df81e4dedfcfe482aa052246d5db536
I20260927 10:49:25.785955 19662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b: Bootstrap starting.
I20260927 10:49:25.788700 19662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:25.789466 19662 log.cc:824] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:25.791342 19662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b: No bootstrap required, opened a new log
Sep 27 10:49:25 dist-test-slave-p9q6 krb5kdc[19581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:49:25.794520 19662 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed69681de1a147e380e72f25dedfef5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42017 } }
I20260927 10:49:25.794818 19662 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:25.794894 19662 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed69681de1a147e380e72f25dedfef5b, State: Initialized, Role: FOLLOWER
I20260927 10:49:25.795354 19662 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed69681de1a147e380e72f25dedfef5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42017 } }
I20260927 10:49:25.795495 19662 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:49:25.795558 19662 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:49:25.795648 19662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:25.796459 19662 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed69681de1a147e380e72f25dedfef5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42017 } }
I20260927 10:49:25.796789 19662 leader_election.cc:302] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed69681de1a147e380e72f25dedfef5b; no voters:
I20260927 10:49:25.797127 19662 leader_election.cc:288] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:49:25.797204 19667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:25.797437 19667 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [term 1 LEADER]: Becoming Leader. State: Replica: ed69681de1a147e380e72f25dedfef5b, State: Running, Role: LEADER
I20260927 10:49:25.797792 19667 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed69681de1a147e380e72f25dedfef5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42017 } }
I20260927 10:49:25.798169 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:25.784161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.9.39.190:42017)
0927 10:49:25.784579 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:25.784599 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:25.784641 (+ 42us) server_negotiation.cc:407] Connection header received
0927 10:49:25.785497 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:25.785519 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:25.785919 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:25.786284 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:25.786958 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:25.788400 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:25.789359 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:25.790023 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:25.790204 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:25.792668 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:25.792674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:25.792687 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:25.792712 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:25.795120 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:25.795581 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:25.795589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:25.795595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:25.795778 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:25.796042 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:25.796046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:25.796048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:25.796426 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:25.796637 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:25.796873 (+ 236us) server_negotiation.cc:299] Negotiation successful
0927 10:49:25.797166 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":108,"threads_started":1}
I20260927 10:49:25.798309 19662 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:49:25.799115 19667 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed69681de1a147e380e72f25dedfef5b. Latest consensus state: current_term: 1 leader_uuid: "ed69681de1a147e380e72f25dedfef5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed69681de1a147e380e72f25dedfef5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42017 } } }
I20260927 10:49:25.799352 19667 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:25.799590 19668 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed69681de1a147e380e72f25dedfef5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed69681de1a147e380e72f25dedfef5b" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42017 } } }
I20260927 10:49:25.799799 19668 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b [sys.catalog]: This master's current role is: LEADER
I20260927 10:49:25.800470 19676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:49:25.803530 19676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:49:25.809373 19676 catalog_manager.cc:1347] Generated new cluster ID: ac059f599b8a4fb18294b878284937ec
I20260927 10:49:25.809441 19676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:49:25.823249 19676 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:49:25.824486 19676 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:49:25.837194 19676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b: Generated new TSK 0
I20260927 10:49:25.837949 19676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:49:25.874030 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42017
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:25.995638 19688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:25.995900 19688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:25.995968 19688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:26.000186 19688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:26.000267 19688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:26.000366 19688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:49:26.005534 19688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:26.006548 19688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:26.007587 19688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:26.017294 19693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:26.017342 19694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:26.017333 19688 server_base.cc:1034] running on GCE node
W20260927 10:49:26.017413 19696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:26.017889 19688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:26.018523 19688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:26.019672 19688 hybrid_clock.cc:648] HybridClock initialized: now 1790506166019648 us; error 71 us; skew 500 ppm
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[19581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:26.022382 19688 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:49:26.023264 19688 webserver.cc:460] Webserver started at http://127.9.39.129:43351/ using document root <none> and password file <none>
I20260927 10:49:26.023806 19688 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:26.023891 19688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:26.024106 19688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:26.026083 19688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "41fd599201231129e440c39c5e0e49eb"
server_key_iv: "9fcf0d69614938ef852577280ecb3a37"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.026638 19688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "41fd599201231129e440c39c5e0e49eb"
server_key_iv: "9fcf0d69614938ef852577280ecb3a37"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.030123 19688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:49:26.032469 19703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.033617 19688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:26.033742 19688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "41fd599201231129e440c39c5e0e49eb"
server_key_iv: "9fcf0d69614938ef852577280ecb3a37"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.033859 19688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:26.054822 19688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:26.058161 19688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:26.058370 19688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:26.058997 19688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:26.059834 19688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:26.059901 19688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.059967 19688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:26.060015 19688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.068682 19688 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:33549
I20260927 10:49:26.068761 19816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:33549 every 8 connection(s)
I20260927 10:49:26.069609 19688 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:49:26.071017 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19688
I20260927 10:49:26.071130 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:49:26.071473 9374 external_mini_cluster.cc:1428] Setting key 6bd773b82b093b03ce6ae9b6742463c1
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[19581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:49:26.084318 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.071563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:57523 (local address 127.9.39.190:42017)
0927 10:49:26.071703 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.071707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.072623 (+ 916us) server_negotiation.cc:407] Connection header received
0927 10:49:26.074594 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.074596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.074639 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.074889 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:26.076041 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.076581 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.077778 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.078027 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.078134 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.080577 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:26.080583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:26.080589 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:26.080624 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:26.082103 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.082591 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.082594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.082595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.082672 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.083063 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.083067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.083069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.083298 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:26.083409 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.083907 (+ 498us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.084074 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 10:49:26.084996 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.072024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42017 (local address 127.9.39.129:57523)
0927 10:49:26.072478 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.072520 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.074352 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.075057 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.075064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.075415 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:26.075895 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.075909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.076705 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.076708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.077657 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.077666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.078174 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.078177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.078344 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.078980 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:26.078995 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:26.080412 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:26.082212 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:26.082218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:26.082230 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:26.082494 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:26.082813 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:26.082815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:26.082817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:26.082954 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:26.083422 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:26.083427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:26.083659 (+ 232us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.083830 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.084096 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":141,"threads_started":1}
I20260927 10:49:26.086350 19817 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42017
I20260927 10:49:26.086604 19817 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:26.087113 19817 heartbeater.cc:511] Master 127.9.39.190:42017 requested a full tablet report, sending...
I20260927 10:49:26.088809 19626 ts_manager.cc:194] Registered new tserver with Master: fce7175ad15f4c3ca1d3af68ad74b9b4 (127.9.39.129:33549)
I20260927 10:49:26.090440 19626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:57523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:49:26.099921 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42017
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:26.207185 19822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:26.207468 19822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:26.207512 19822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:26.211431 19822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:26.211480 19822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:26.211536 19822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:49:26.216856 19822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:26.217797 19822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:26.218771 19822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:26.228123 19830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:26.228117 19827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:26.228328 19822 server_base.cc:1034] running on GCE node
W20260927 10:49:26.228119 19828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:26.228909 19822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:26.229492 19822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:26.230669 19822 hybrid_clock.cc:648] HybridClock initialized: now 1790506166230650 us; error 32 us; skew 500 ppm
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[19581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:26.233347 19822 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:49:26.234231 19822 webserver.cc:460] Webserver started at http://127.9.39.130:37041/ using document root <none> and password file <none>
I20260927 10:49:26.234822 19822 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:26.234908 19822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:26.235127 19822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:26.237136 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "d9e55191e43a40c0abb3dc532ccfdb3a"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "a76cf1223d6f22465b883ea63d17158b"
server_key_iv: "1fe9fd40da92afaff5c8cbe7d177e0ae"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.237736 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "d9e55191e43a40c0abb3dc532ccfdb3a"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "a76cf1223d6f22465b883ea63d17158b"
server_key_iv: "1fe9fd40da92afaff5c8cbe7d177e0ae"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.241317 19822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 10:49:26.244010 19837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.245203 19822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:49:26.245335 19822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "d9e55191e43a40c0abb3dc532ccfdb3a"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "a76cf1223d6f22465b883ea63d17158b"
server_key_iv: "1fe9fd40da92afaff5c8cbe7d177e0ae"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.245458 19822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:26.260118 19822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:26.262809 19822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:26.262981 19822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:26.263518 19822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:26.264288 19822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:26.264353 19822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.264418 19822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:26.264464 19822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.273317 19822 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:33209
I20260927 10:49:26.273345 19950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:33209 every 8 connection(s)
I20260927 10:49:26.274164 19822 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:49:26.275956 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19822
I20260927 10:49:26.276082 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:49:26.276382 9374 external_mini_cluster.cc:1428] Setting key 8d46db081745086c71a2148c173d3fa1
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[19581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:49:26.289510 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.275956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:41985 (local address 127.9.39.190:42017)
0927 10:49:26.276112 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.276117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.277025 (+ 908us) server_negotiation.cc:407] Connection header received
0927 10:49:26.278800 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.278803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.278844 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.278913 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:26.280326 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.281068 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.282878 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.283086 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.283163 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.285953 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:26.285956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:26.285958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:26.285979 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:26.287340 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.287868 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.287876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.287877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.287963 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.288313 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.288316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.288318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.288575 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:26.288679 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.289211 (+ 532us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.289327 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:49:26.290395 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.276379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42017 (local address 127.9.39.130:41985)
0927 10:49:26.276876 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.276917 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.278539 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.279089 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.279100 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.279637 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:26.280187 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.280200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.281203 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.281210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.282764 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.282779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.283246 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.283254 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.283506 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.284133 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:26.284154 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:26.285791 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:26.287447 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:26.287453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:26.287468 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:26.287764 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:26.288048 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:26.288053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:26.288055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:26.288198 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:26.288677 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:26.288684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:26.288944 (+ 260us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.289141 (+ 197us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.289473 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":98,"threads_started":1}
I20260927 10:49:26.291846 19951 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42017
I20260927 10:49:26.292099 19951 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:26.292560 19951 heartbeater.cc:511] Master 127.9.39.190:42017 requested a full tablet report, sending...
I20260927 10:49:26.293596 19626 ts_manager.cc:194] Registered new tserver with Master: d9e55191e43a40c0abb3dc532ccfdb3a (127.9.39.130:33209)
I20260927 10:49:26.294148 19626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:41985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:49:26.305692 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42017
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:49:26.423048 19956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:49:26.423282 19956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:49:26.423323 19956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:49:26.427502 19956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:49:26.427554 19956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:49:26.427610 19956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:49:26.432636 19956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:49:26.433552 19956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:49:26.434517 19956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:49:26.443918 19961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:49:26.443964 19962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:26.444073 19956 server_base.cc:1034] running on GCE node
W20260927 10:49:26.443924 19964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:49:26.444720 19956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:49:26.445286 19956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:49:26.446427 19956 hybrid_clock.cc:648] HybridClock initialized: now 1790506166446399 us; error 44 us; skew 500 ppm
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[19581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:49:26.449162 19956 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:49:26.450013 19956 webserver.cc:460] Webserver started at http://127.9.39.131:35049/ using document root <none> and password file <none>
I20260927 10:49:26.450536 19956 fs_manager.cc:359] Metadata directory not provided
I20260927 10:49:26.450592 19956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:49:26.450839 19956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:49:26.452786 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "f9929031ba7e43468a5f34385382dcb0"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "f31bd196aea78f1c692c5253edf019fd"
server_key_iv: "68256a156241d73c57d0458c23aa963e"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.453311 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "f9929031ba7e43468a5f34385382dcb0"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "f31bd196aea78f1c692c5253edf019fd"
server_key_iv: "68256a156241d73c57d0458c23aa963e"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.456897 19956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:49:26.459260 19971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.460418 19956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:49:26.460529 19956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "f9929031ba7e43468a5f34385382dcb0"
format_stamp: "Formatted at 2026-09-27 10:49:26 on dist-test-slave-p9q6"
server_key: "f31bd196aea78f1c692c5253edf019fd"
server_key_iv: "68256a156241d73c57d0458c23aa963e"
server_key_version: "encryptionkey@0"
I20260927 10:49:26.460628 19956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:49:26.476269 19956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:49:26.479184 19956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:49:26.479387 19956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:49:26.479962 19956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:49:26.480746 19956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:49:26.480792 19956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.480830 19956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:49:26.480845 19956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:49:26.489334 19956 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:36783
I20260927 10:49:26.489365 20084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:36783 every 8 connection(s)
I20260927 10:49:26.490171 19956 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:49:26.493244 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 19956
I20260927 10:49:26.493404 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:49:26.493834 9374 external_mini_cluster.cc:1428] Setting key d931fbbc848da53643067879c7da33d7
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[19581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:49:26.503638 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.491918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:38311 (local address 127.9.39.190:42017)
0927 10:49:26.492082 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.492087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.492882 (+ 795us) server_negotiation.cc:407] Connection header received
0927 10:49:26.494582 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.494585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.494641 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.494782 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:26.496237 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.496783 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.497928 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.498126 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.498204 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.500629 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:26.500631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:26.500634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:26.500655 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:26.501867 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.502331 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.502334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.502335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.502402 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.502742 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.502747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.502749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.502918 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:26.502994 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.503434 (+ 440us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.503524 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 10:49:26.504477 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.492290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:42017 (local address 127.9.39.131:38311)
0927 10:49:26.492730 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.492774 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.494389 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.494960 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.494969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.495475 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:49:26.496097 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.496122 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.496905 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.496908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.497825 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.497835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.498254 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.498257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.498397 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.499034 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:49:26.499051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:49:26.500489 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:49:26.501966 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:26.501971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:26.501982 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:26.502240 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:26.502475 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:49:26.502477 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:49:26.502479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:49:26.502621 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:49:26.503003 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:49:26.503008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:49:26.503232 (+ 224us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.503398 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.503696 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":94,"threads_started":1}
I20260927 10:49:26.505594 20085 heartbeater.cc:347] Connected to a master server at 127.9.39.190:42017
I20260927 10:49:26.505842 20085 heartbeater.cc:464] Registering TS with master...
I20260927 10:49:26.506414 20085 heartbeater.cc:511] Master 127.9.39.190:42017 requested a full tablet report, sending...
I20260927 10:49:26.507552 19625 ts_manager.cc:194] Registered new tserver with Master: f9929031ba7e43468a5f34385382dcb0 (127.9.39.131:36783)
I20260927 10:49:26.508183 19625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:38311
I20260927 10:49:26.519495 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:49:26.528262 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.521443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.9.39.190:42017)
0927 10:49:26.521602 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.521606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.521756 (+ 150us) server_negotiation.cc:407] Connection header received
0927 10:49:26.521863 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.521865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.521904 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.521971 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:49:26.522657 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.523220 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.524058 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.524261 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.524332 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.525650 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:49:26.525652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:49:26.525654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:49:26.525673 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:49:26.526890 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.527246 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.527248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.527249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.527315 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:49:26.527520 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:49:26.527522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:49:26.527523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:49:26.527712 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:49:26.527806 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.527978 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.528061 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 10:49:26.531364 19625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:49:26.534433 19625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:49:26.548290 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.542428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:36783 (local address 127.0.0.1:33442)
0927 10:49:26.542843 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.542894 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.543252 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.543863 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.543869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.543891 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:26.544032 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.544038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.545427 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.545430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.546452 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.546462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.547744 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.547748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.547862 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.547925 (+ 63us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.548033 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.548142 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260927 10:49:26.548391 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.542573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.9.39.131:36783)
0927 10:49:26.543071 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.543084 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.543406 (+ 322us) server_negotiation.cc:407] Connection header received
0927 10:49:26.543493 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.543502 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.543709 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.543867 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:26.544159 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.545302 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.546591 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.547428 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.547526 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.547678 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.548078 (+ 400us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.548179 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":180,"threads_started":1}
I20260927 10:49:26.548969 20099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.543190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:33209 (local address 127.0.0.1:58096)
0927 10:49:26.544124 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.544136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.544232 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.544756 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.544759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.544776 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:26.544867 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.544873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.546162 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.546165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.547397 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.547408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.548567 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.548573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.548675 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.548702 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.548765 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.548841 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":46,"thread_start_us":78,"threads_started":1}
I20260927 10:49:26.549147 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.542830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:33549 (local address 127.0.0.1:39218)
0927 10:49:26.543372 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.543387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.543462 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.545034 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.545037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.545053 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:26.545151 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.545156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.546520 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.546523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.547579 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.547590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.548784 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.548787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.548862 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.548879 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.548964 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.549030 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260927 10:49:26.549211 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.543255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.9.39.129:33549)
0927 10:49:26.544449 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.544463 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.544484 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:49:26.544554 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.544559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.544756 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.544895 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:26.545269 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.546359 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.547696 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.548499 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.548595 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.548724 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.548993 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.549081 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":58,"threads_started":1}
I20260927 10:49:26.549480 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.543585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.9.39.130:33209)
0927 10:49:26.543859 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.543872 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.544331 (+ 459us) server_negotiation.cc:407] Connection header received
0927 10:49:26.544467 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.544473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.544633 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.544785 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:26.544967 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.546031 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.547526 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.548295 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.548466 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.549204 (+ 738us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.549251 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.549327 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
I20260927 10:49:26.551754 20019 tablet_service.cc:1467] Processing CreateTablet for tablet 714d46eae60047dc843d08e749fbb857 (DEFAULT_TABLE table=test-table [id=fc01fa6171e54e95a0d1eff00e60dc2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:26.551748 19751 tablet_service.cc:1467] Processing CreateTablet for tablet 714d46eae60047dc843d08e749fbb857 (DEFAULT_TABLE table=test-table [id=fc01fa6171e54e95a0d1eff00e60dc2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:26.552683 20019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714d46eae60047dc843d08e749fbb857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:26.552713 19751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714d46eae60047dc843d08e749fbb857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:26.553368 19885 tablet_service.cc:1467] Processing CreateTablet for tablet 714d46eae60047dc843d08e749fbb857 (DEFAULT_TABLE table=test-table [id=fc01fa6171e54e95a0d1eff00e60dc2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:49:26.554287 19885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714d46eae60047dc843d08e749fbb857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:49:26.559455 20103 tablet_bootstrap.cc:492] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4: Bootstrap starting.
I20260927 10:49:26.560240 20102 tablet_bootstrap.cc:492] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Bootstrap starting.
I20260927 10:49:26.560732 20104 tablet_bootstrap.cc:492] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Bootstrap starting.
I20260927 10:49:26.561757 20103 tablet_bootstrap.cc:654] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:26.562855 20102 tablet_bootstrap.cc:654] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:26.562883 20104 tablet_bootstrap.cc:654] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Neither blocks nor log segments found. Creating new log.
I20260927 10:49:26.563227 20103 log.cc:824] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:26.563604 20104 log.cc:824] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:26.563697 20102 log.cc:824] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Log is configured to *not* fsync() on all Append() calls
I20260927 10:49:26.565593 20102 tablet_bootstrap.cc:492] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: No bootstrap required, opened a new log
I20260927 10:49:26.565670 20103 tablet_bootstrap.cc:492] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4: No bootstrap required, opened a new log
I20260927 10:49:26.565665 20104 tablet_bootstrap.cc:492] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: No bootstrap required, opened a new log
I20260927 10:49:26.565800 20102 ts_tablet_manager.cc:1397] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260927 10:49:26.565840 20103 ts_tablet_manager.cc:1397] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260927 10:49:26.565840 20104 ts_tablet_manager.cc:1397] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260927 10:49:26.568614 20102 raft_consensus.cc:348] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.568847 20102 raft_consensus.cc:374] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:26.568917 20102 raft_consensus.cc:729] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9929031ba7e43468a5f34385382dcb0, State: Initialized, Role: FOLLOWER
I20260927 10:49:26.569386 20102 consensus_queue.cc:260] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.569564 20103 raft_consensus.cc:348] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.569564 20104 raft_consensus.cc:348] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.569806 20104 raft_consensus.cc:374] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:26.569880 20103 raft_consensus.cc:374] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:49:26.569939 20103 raft_consensus.cc:729] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce7175ad15f4c3ca1d3af68ad74b9b4, State: Initialized, Role: FOLLOWER
I20260927 10:49:26.570034 20085 heartbeater.cc:503] Master 127.9.39.190:42017 was elected leader, sending a full tablet report...
I20260927 10:49:26.570372 20102 ts_tablet_manager.cc:1428] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260927 10:49:26.570358 20103 consensus_queue.cc:260] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.570906 20104 raft_consensus.cc:729] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e55191e43a40c0abb3dc532ccfdb3a, State: Initialized, Role: FOLLOWER
I20260927 10:49:26.571393 20103 ts_tablet_manager.cc:1428] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 10:49:26.571396 20104 consensus_queue.cc:260] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.571745 19817 heartbeater.cc:503] Master 127.9.39.190:42017 was elected leader, sending a full tablet report...
I20260927 10:49:26.572021 19951 heartbeater.cc:503] Master 127.9.39.190:42017 was elected leader, sending a full tablet report...
I20260927 10:49:26.572310 20104 ts_tablet_manager.cc:1428] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260927 10:49:26.722055 20110 raft_consensus.cc:482] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:49:26.722270 20110 raft_consensus.cc:504] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.723400 20110 leader_election.cc:288] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9929031ba7e43468a5f34385382dcb0 (127.9.39.131:36783), fce7175ad15f4c3ca1d3af68ad74b9b4 (127.9.39.129:33549)
I20260927 10:49:26.728519 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.723973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:41625 (local address 127.9.39.131:36783)
0927 10:49:26.724102 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.724108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.724247 (+ 139us) server_negotiation.cc:407] Connection header received
0927 10:49:26.724319 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.724322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.724375 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.724445 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:26.724761 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.725554 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.727340 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.727908 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.728039 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.728089 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.728345 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.728412 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:49:26.728536 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.723718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:36783 (local address 127.9.39.130:41625)
0927 10:49:26.724144 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.724156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.724240 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.724462 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.724464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.724476 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:26.724604 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.724613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.725674 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.725678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.727221 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.727227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.728039 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.728043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.728184 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.728224 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.728307 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.728387 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":188,"thread_start_us":130,"threads_started":1}
I20260927 10:49:26.728580 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.723718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:33549 (local address 127.9.39.130:39945)
0927 10:49:26.723929 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.723945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:49:26.724058 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:49:26.724385 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:49:26.724386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:49:26.724398 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:49:26.724564 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.724573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.725674 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.725683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.727207 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:49:26.727219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.728155 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.728159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:49:26.728293 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.728315 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:49:26.728387 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 10:49:26.728459 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":125}
I20260927 10:49:26.728579 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.723808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:39945 (local address 127.9.39.129:33549)
0927 10:49:26.723930 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.723935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.724060 (+ 125us) server_negotiation.cc:407] Connection header received
0927 10:49:26.724185 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.724188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.724267 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.724359 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:49:26.724761 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.725554 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.727367 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.727918 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.728030 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.728089 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.728403 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.728468 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:49:26.729298 20039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714d46eae60047dc843d08e749fbb857" candidate_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9929031ba7e43468a5f34385382dcb0" is_pre_election: true
I20260927 10:49:26.729384 19771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714d46eae60047dc843d08e749fbb857" candidate_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" is_pre_election: true
I20260927 10:49:26.729619 20039 raft_consensus.cc:2454] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9e55191e43a40c0abb3dc532ccfdb3a in term 0.
I20260927 10:49:26.729691 19771 raft_consensus.cc:2454] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9e55191e43a40c0abb3dc532ccfdb3a in term 0.
I20260927 10:49:26.730099 19840 leader_election.cc:302] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9e55191e43a40c0abb3dc532ccfdb3a, f9929031ba7e43468a5f34385382dcb0; no voters:
I20260927 10:49:26.730402 20110 raft_consensus.cc:2781] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:49:26.730487 20110 raft_consensus.cc:482] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:49:26.730520 20110 raft_consensus.cc:3037] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:26.731614 20110 raft_consensus.cc:504] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.731994 20110 leader_election.cc:288] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [CANDIDATE]: Term 1 election: Requested vote from peers f9929031ba7e43468a5f34385382dcb0 (127.9.39.131:36783), fce7175ad15f4c3ca1d3af68ad74b9b4 (127.9.39.129:33549)
I20260927 10:49:26.732360 19771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714d46eae60047dc843d08e749fbb857" candidate_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4"
I20260927 10:49:26.732343 20039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714d46eae60047dc843d08e749fbb857" candidate_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9929031ba7e43468a5f34385382dcb0"
I20260927 10:49:26.732488 19771 raft_consensus.cc:3037] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:26.732496 20039 raft_consensus.cc:3037] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:49:26.733472 19771 raft_consensus.cc:2454] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9e55191e43a40c0abb3dc532ccfdb3a in term 1.
I20260927 10:49:26.733539 20039 raft_consensus.cc:2454] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9e55191e43a40c0abb3dc532ccfdb3a in term 1.
I20260927 10:49:26.733759 19839 leader_election.cc:302] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9e55191e43a40c0abb3dc532ccfdb3a, fce7175ad15f4c3ca1d3af68ad74b9b4; no voters:
I20260927 10:49:26.733913 20110 raft_consensus.cc:2781] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:49:26.734155 20110 raft_consensus.cc:686] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 1 LEADER]: Becoming Leader. State: Replica: d9e55191e43a40c0abb3dc532ccfdb3a, State: Running, Role: LEADER
I20260927 10:49:26.734503 20110 consensus_queue.cc:237] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } }
I20260927 10:49:26.737824 19625 catalog_manager.cc:5495] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a reported cstate change: term changed from 0 to 1, leader changed from <none> to d9e55191e43a40c0abb3dc532ccfdb3a (127.9.39.130). New cstate: current_term: 1 leader_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9e55191e43a40c0abb3dc532ccfdb3a" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 } health_report { overall_health: UNKNOWN } } }
W20260927 10:49:26.741106 20086 tablet.cc:2369] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:49:26.775305 19952 tablet.cc:2369] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:49:26.782577 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.778733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.9.39.130:33209)
0927 10:49:26.778854 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 10:49:26.778857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:49:26.778926 (+ 69us) server_negotiation.cc:407] Connection header received
0927 10:49:26.779046 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:49:26.779050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:49:26.779117 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:49:26.779237 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:49:26.779451 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.780195 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.781023 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:49:26.781374 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:49:26.781469 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:49:26.781724 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:49:26.782197 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:49:26.782312 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 10:49:26.782369 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 10:49:26.782425 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 10:49:26.789142 20039 raft_consensus.cc:1261] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 1 FOLLOWER]: Refusing update from remote peer d9e55191e43a40c0abb3dc532ccfdb3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:26.789149 19771 raft_consensus.cc:1261] T 714d46eae60047dc843d08e749fbb857 P fce7175ad15f4c3ca1d3af68ad74b9b4 [term 1 FOLLOWER]: Refusing update from remote peer d9e55191e43a40c0abb3dc532ccfdb3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:49:26.789906 20110 consensus_queue.cc:1035] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9929031ba7e43468a5f34385382dcb0" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:26.790112 20112 consensus_queue.cc:1035] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fce7175ad15f4c3ca1d3af68ad74b9b4" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 33549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:49:26.796077 20115 mvcc.cc:205] Tried to move back new op lower bound from 7333913259161116672 to 7333913258947366912. Current Snapshot: MvccSnapshot[applied={T|T < 7333913259161116672}]
I20260927 10:49:26.798126 20117 mvcc.cc:205] Tried to move back new op lower bound from 7333913259161116672 to 7333913258947366912. Current Snapshot: MvccSnapshot[applied={T|T < 7333913259161116672}]
I20260927 10:49:26.800907 20116 mvcc.cc:205] Tried to move back new op lower bound from 7333913259161116672 to 7333913258947366912. Current Snapshot: MvccSnapshot[applied={T|T < 7333913259161116672 or (T in {7333913259161116672})}]
I20260927 10:49:26.805011 19626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:26.805145 19626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:49:26.807502 19626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed69681de1a147e380e72f25dedfef5b: Sending DeleteTablet for 3 replicas of tablet 714d46eae60047dc843d08e749fbb857
I20260927 10:49:26.808300 20019 tablet_service.cc:1514] Processing DeleteTablet for tablet 714d46eae60047dc843d08e749fbb857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:26 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:33442
I20260927 10:49:26.808310 19885 tablet_service.cc:1514] Processing DeleteTablet for tablet 714d46eae60047dc843d08e749fbb857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:26 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:58096
I20260927 10:49:26.808794 20124 tablet_replica.cc:330] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: stopping tablet replica
I20260927 10:49:26.808791 20125 tablet_replica.cc:330] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: stopping tablet replica
I20260927 10:49:26.808809 19751 tablet_service.cc:1514] Processing DeleteTablet for tablet 714d46eae60047dc843d08e749fbb857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:49:26 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:39218
I20260927 10:49:26.809092 20125 raft_consensus.cc:2229] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:49:26.809092 20124 raft_consensus.cc:2229] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:49:26.809379 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19688
I20260927 10:49:26.809382 20125 raft_consensus.cc:2258] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:26.809502 20124 raft_consensus.cc:2258] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:49:26.815836 20125 ts_tablet_manager.cc:1905] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 10:49:26.815829 19611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:33549 (error 108)
I20260927 10:49:26.815922 20124 ts_tablet_manager.cc:1905] T 714d46eae60047dc843d08e749fbb857 P d9e55191e43a40c0abb3dc532ccfdb3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:49:26.817206 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.816878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:33549 (local address 127.0.0.1:39234)
0927 10:49:26.817053 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.817117 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:33549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260927 10:49:26.817458 19611 catalog_manager.cc:4584] TS fce7175ad15f4c3ca1d3af68ad74b9b4 (127.9.39.129:33549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 714d46eae60047dc843d08e749fbb857: Network error: Client connection negotiation failed: client connection to 127.9.39.129:33549: connect: Connection refused (error 111)
I20260927 10:49:26.817876 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19822
I20260927 10:49:26.820449 20125 ts_tablet_manager.cc:1918] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:49:26.820571 20125 log.cc:1196] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/714d46eae60047dc843d08e749fbb857
I20260927 10:49:26.820887 20125 ts_tablet_manager.cc:1939] T 714d46eae60047dc843d08e749fbb857 P f9929031ba7e43468a5f34385382dcb0: Deleting consensus metadata
I20260927 10:49:26.822261 19612 catalog_manager.cc:4841] TS f9929031ba7e43468a5f34385382dcb0 (127.9.39.131:36783): tablet 714d46eae60047dc843d08e749fbb857 (table test-table [id=fc01fa6171e54e95a0d1eff00e60dc2b]) successfully deleted
I20260927 10:49:26.825598 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19956
I20260927 10:49:26.826483 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:49:26.825751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:33209 (local address 127.0.0.1:58108)
0927 10:49:26.826314 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0927 10:49:26.826380 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.130:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456}
I20260927 10:49:26.831172 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 19594
2026-09-27T10:49:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2382 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[20130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[20130](info): set up 2 sockets
Sep 27 10:49:26 dist-test-slave-p9q6 krb5kdc[20130](info): commencing operation
krb5kdc: starting...
W20260927 10:49:27.883961 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:49:27 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:49:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/rangeradmin_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/rangeradmin_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/rangerlookup_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/rangerlookup_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
I20260927 10:49:28.053166 9374 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasku3mICI/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/postgres -l logfile start
2026-09-27 10:49:30.506 UTC [20160] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 10:49:30.506 UTC [20160] LOG: listening on IPv4 address "127.9.39.148", port 49933
2026-09-27 10:49:30.510 UTC [20160] LOG: listening on Unix socket "/tmp/.s.PGSQL.49933"
2026-09-27 10:49:30.545 UTC [20163] LOG: database system was shut down at 2026-09-27 10:49:29 UTC
2026-09-27 10:49:30.549 UTC [20160] LOG: database system is ready to accept connections
I20260927 10:49:31.499399 9374 mini_postgres.cc:96] Postgres bound to 49933
2026-09-27 10:49:31.505 UTC [20171] FATAL: database "slave" does not exist
127.9.39.148:49933 - accepting connections
I20260927 10:49:31.506305 9374 mini_ranger.cc:161] Starting Ranger...
I20260927 10:49:31.520583 9374 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 10:49:31.985966 9374 mini_ranger.cc:87] Created ranger Postgres database
I20260927 10:49:31.986068 9374 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-admin
2026-09-27 10:49:32,253 [I] DB FLAVOR :POSTGRES
2026-09-27 10:49:32,253 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 10:49:32,254 [I] Checking connection..
2026-09-27 10:49:32,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 10:49:32,492 [I] Checking connection passed.
2026-09-27 10:49:32,493 [I] --------- Verifying version history table ---------
2026-09-27 10:49:32,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 10:49:32,723 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 10:49:32,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 10:49:32,975 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 10:49:32,975 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 10:49:32,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 10:49:33.237 UTC [20235] WARNING: there is no transaction in progress
2026-09-27 10:49:33,243 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 10:49:33,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 10:49:33,473 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 10:49:33,474 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 10:49:33,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 10:49:33,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 10:49:33,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p9q6.c.gcp-upstream.internal','N') ;"
2026-09-27 10:49:34,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 10:49:34,398 [I] Table x_portal_user does not exist in database ranger
2026-09-27 10:49:34,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 10:49:34,629 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 10:49:34,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 10:49:34,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 10:49:35,108 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 10:49:35,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 10:49:36.033 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.050 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.067 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.083 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.098 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.114 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.166 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.172 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.187 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.196 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.206 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.220 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.231 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.238 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.244 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36.251 UTC [20343] WARNING: there is no transaction in progress
2026-09-27 10:49:36,867 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 10:49:36,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 10:49:37,120 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 10:49:37,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 10:49:37,352 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 10:49:37,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 10:49:37,603 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 10:49:37,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 10:49:37,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 10:49:38,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p9q6.c.gcp-upstream.internal';"
2026-09-27 10:49:38,339 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 10:49:38,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 10:49:38,564 [I] DB_PATCHES have already been applied
I20260927 10:49:38.570678 9374 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-admin:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasku3mICI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 10:49:38.570781 9374 mini_ranger.cc:193] Using host: 127.9.39.148
I20260927 10:49:38.574087 9374 mini_ranger.cc:227] Ranger admin URL: http://127.9.39.148:57761
Sep 27, 2026 10:49:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 10:49:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 10:49:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 10:49:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 10:49:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 10:49:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 10:49:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 10:49:39.682937 9374 mini_ranger.cc:160] Time spent starting Ranger: real 8.177s user 0.000s sys 0.009s
Sep 27, 2026 10:49:59 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 10:49:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 27, 2026 10:49:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 10:49:59 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 10:50:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 10:50:00.942457 9374 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/rangerkms_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/rangerkms_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.39.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.39.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 10:50:01 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506201, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 10:50:01.023409 9374 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 10:50:01.032495 9374 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 10:50:01.844703 9374 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 10:50:01.844803 9374 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms
I20260927 10:50:01.844905 9374 mini_ranger_kms.cc:227] Using postgres at 127.9.39.148:49933
2026-09-27 10:50:02,019 [I] DB FLAVOR :POSTGRES
2026-09-27 10:50:02,020 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 10:50:02,020 [I] Checking connection
2026-09-27 10:50:02,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 10:50:02,243 [I] connection success
2026-09-27 10:50:02,243 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 10:50:02,243 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 10:50:02,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 10:50:02,510 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 10:50:02,510 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 10:50:02,511 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 10:50:02,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:49933/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 10:50:02,756 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 10:50:03.223055 9374 mini_ranger_kms.cc:310] Created kms service
I20260927 10:50:03.389957 9374 mini_ranger_kms.cc:326] Created kudu user
I20260927 10:50:03.430377 9374 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 10:50:03 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506201, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:03.791545 9374 mini_ranger_kms.cc:384] Added ranger policy
I20260927 10:50:03.791690 9374 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms:/tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 10:50:03.791713 9374 mini_ranger_kms.cc:240] Using host: 127.9.39.148
I20260927 10:50:03.795841 9374 mini_ranger_kms.cc:276] Ranger KMS PID: 20523
I20260927 10:50:03.795964 9374 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.39.148:45655
Sep 27, 2026 10:50:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 10:50:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 10:50:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 10:50:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 10:50:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 10:50:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 10:50:06.058745 9374 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.035s user 0.015s sys 0.144s
I20260927 10:50:06.058861 9374 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 10:50:06.058956 9374 mini_ranger_kms.cc:401] 127.9.39.148:45655/kms/v1/keys
Sep 27 10:50:06 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790506206, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.39.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 10:50:06 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506206, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.39.148@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
Sep 27 10:50:07 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506206, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.39.148@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
Sep 27 10:50:09 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:50:09.221738 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:41631
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:41631
--ranger_config_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:45655/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:50:09.326144 20574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:09.326380 20574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:09.326416 20574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:09.330395 20574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:50:09.330446 20574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:09.330464 20574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:09.330520 20574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:50:09.330551 20574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:50:09.335728 20574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.39.148:45655/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:41631
--ranger_config_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:41631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:09.336701 20574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:09.337649 20574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:09.346308 20580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:09.346433 20574 server_base.cc:1034] running on GCE node
W20260927 10:50:09.346334 20582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:09.346694 20579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:09.347286 20574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:09.348290 20574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:09.349481 20574 hybrid_clock.cc:648] HybridClock initialized: now 1790506209349465 us; error 34 us; skew 500 ppm
Sep 27 10:50:09 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506209, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:09.357566 20574 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:50:09.358415 20574 webserver.cc:460] Webserver started at http://127.9.39.190:42867/ using document root <none> and password file <none>
I20260927 10:50:09.359010 20574 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:09.359086 20574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:09.359316 20574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:09 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506209, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:09.403716 20574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "d72e2241f99c47f4b8c91d498064cc2a"
format_stamp: "Formatted at 2026-09-27 10:50:09 on dist-test-slave-p9q6"
server_key: "23f6fc08ba8c9698e5f9509cdb7c1215"
server_key_iv: "d3fc8407a566ae6610289efc2bc602e7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:09.404297 20574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "d72e2241f99c47f4b8c91d498064cc2a"
format_stamp: "Formatted at 2026-09-27 10:50:09 on dist-test-slave-p9q6"
server_key: "23f6fc08ba8c9698e5f9509cdb7c1215"
server_key_iv: "d3fc8407a566ae6610289efc2bc602e7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:09.408178 20574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:50:09.420888 20589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:09.422216 20574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:50:09.422351 20574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "d72e2241f99c47f4b8c91d498064cc2a"
format_stamp: "Formatted at 2026-09-27 10:50:09 on dist-test-slave-p9q6"
server_key: "23f6fc08ba8c9698e5f9509cdb7c1215"
server_key_iv: "d3fc8407a566ae6610289efc2bc602e7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:09.422497 20574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:09.437623 20574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:09.440550 20574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:09.440739 20574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:09.449225 20574 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:41631
I20260927 10:50:09.449266 20641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:41631 every 8 connection(s)
I20260927 10:50:09.450227 20574 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:50:09.453042 20642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:09.458297 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 20574
I20260927 10:50:09.458431 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:50:09.458616 20642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a: Bootstrap starting.
I20260927 10:50:09.458731 9374 external_mini_cluster.cc:1428] Setting key 09dcd62290a6bcb2cfd37ab6f156383f
I20260927 10:50:09.461472 20642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:09.462363 20642 log.cc:824] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:09.464934 20642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a: No bootstrap required, opened a new log
Sep 27 10:50:09 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:09.470057 20642 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72e2241f99c47f4b8c91d498064cc2a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41631 } }
I20260927 10:50:09.470305 20642 raft_consensus.cc:374] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:09.470396 20642 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72e2241f99c47f4b8c91d498064cc2a, State: Initialized, Role: FOLLOWER
I20260927 10:50:09.470938 20642 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72e2241f99c47f4b8c91d498064cc2a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41631 } }
I20260927 10:50:09.471160 20642 raft_consensus.cc:388] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:50:09.471238 20642 raft_consensus.cc:482] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:50:09.471340 20642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:09.472345 20642 raft_consensus.cc:504] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72e2241f99c47f4b8c91d498064cc2a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41631 } }
I20260927 10:50:09.472759 20642 leader_election.cc:302] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d72e2241f99c47f4b8c91d498064cc2a; no voters:
I20260927 10:50:09.473121 20642 leader_election.cc:288] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:50:09.473517 20648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:50:09.473754 20648 raft_consensus.cc:686] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [term 1 LEADER]: Becoming Leader. State: Replica: d72e2241f99c47f4b8c91d498064cc2a, State: Running, Role: LEADER
I20260927 10:50:09.474097 20648 consensus_queue.cc:237] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72e2241f99c47f4b8c91d498064cc2a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41631 } }
I20260927 10:50:09.474463 20642 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:50:09.476049 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:09.460146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.9.39.190:41631)
0927 10:50:09.460595 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:09.460615 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:09.460657 (+ 42us) server_negotiation.cc:407] Connection header received
0927 10:50:09.461569 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:09.461594 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:09.462018 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:09.462392 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:09.463279 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:09.464820 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:09.466072 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:09.466844 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:09.466995 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:09.470409 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:09.470416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:09.470433 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:09.470464 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:09.472585 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:09.473127 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:09.473133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:09.473139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:09.473302 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:09.473621 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:09.473624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:09.473626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:09.474092 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:09.474268 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:09.474750 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 10:50:09.475035 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
I20260927 10:50:09.476756 20642 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 10:50:09.475401 20648 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d72e2241f99c47f4b8c91d498064cc2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72e2241f99c47f4b8c91d498064cc2a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41631 } } }
I20260927 10:50:09.477681 20648 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [sys.catalog]: This master's current role is: LEADER
I20260927 10:50:09.477880 20648 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d72e2241f99c47f4b8c91d498064cc2a. Latest consensus state: current_term: 1 leader_uuid: "d72e2241f99c47f4b8c91d498064cc2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72e2241f99c47f4b8c91d498064cc2a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41631 } } }
I20260927 10:50:09.477972 20648 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a [sys.catalog]: This master's current role is: LEADER
I20260927 10:50:09.481344 20662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:50:09.487783 20662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:50:09.494642 20662 catalog_manager.cc:1347] Generated new cluster ID: b3bed163ab98484a81c637dbbc60021b
I20260927 10:50:09.494726 20662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:50:09.516853 20662 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:50:09.517762 20662 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:50:09.531840 20662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a: Generated new TSK 0
I20260927 10:50:09.532534 20662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:50:10.217818 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:45655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41631
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:50:10.336330 20687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:10.336580 20687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:10.336623 20687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:10.341113 20687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:10.341181 20687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:10.341269 20687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:50:10.348122 20687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.39.148:45655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:10.349735 20687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:10.351080 20687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:10.362368 20692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:10.362486 20695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:10.362377 20693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:10.362907 20687 server_base.cc:1034] running on GCE node
I20260927 10:50:10.363348 20687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:10.364040 20687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:10.365262 20687 hybrid_clock.cc:648] HybridClock initialized: now 1790506210365241 us; error 38 us; skew 500 ppm
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:10.368336 20687 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:50:10.369436 20687 webserver.cc:460] Webserver started at http://127.9.39.129:44219/ using document root <none> and password file <none>
I20260927 10:50:10.370190 20687 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:10.370286 20687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:10.370507 20687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:10.386471 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "ccf47d667037407b829afa28baafa5ec"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "571ad4bf507b2dac641850a40afcf956"
server_key_iv: "1d85a0c4507afd24bdae8507e6a5e8be"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.387308 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "ccf47d667037407b829afa28baafa5ec"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "571ad4bf507b2dac641850a40afcf956"
server_key_iv: "1d85a0c4507afd24bdae8507e6a5e8be"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.396102 20687 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.003s
I20260927 10:50:10.412468 20704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:10.414106 20687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 10:50:10.414222 20687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "ccf47d667037407b829afa28baafa5ec"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "571ad4bf507b2dac641850a40afcf956"
server_key_iv: "1d85a0c4507afd24bdae8507e6a5e8be"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.414341 20687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:10.454685 20687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:10.458287 20687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:10.458503 20687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:10.459247 20687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:10.460305 20687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:10.460404 20687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:10.460517 20687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:10.460592 20687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:10.474328 20687 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:37355
I20260927 10:50:10.475577 20687 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:50:10.476190 20817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:37355 every 8 connection(s)
I20260927 10:50:10.476450 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 20687
I20260927 10:50:10.476583 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:50:10.476892 9374 external_mini_cluster.cc:1428] Setting key 7d30fe957a5107864e327a8e20d6d37c
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:10.497750 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:10.480205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:39035 (local address 127.9.39.190:41631)
0927 10:50:10.482227 (+ 2022us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:10.482238 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:10.484030 (+ 1792us) server_negotiation.cc:407] Connection header received
0927 10:50:10.484227 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:10.484231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:10.484282 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:10.484377 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:10.485820 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.486735 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.488406 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.488756 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.488865 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:10.493003 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:10.493007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:10.493010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:10.493034 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:10.495032 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:10.495690 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:10.495696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:10.495700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:10.495795 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:10.496190 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:10.496194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:10.496196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:10.496430 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:10.497381 (+ 951us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:10.497461 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 10:50:10.497584 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1921,"thread_start_us":94,"threads_started":1}
I20260927 10:50:10.498471 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:10.479176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:41631 (local address 127.9.39.129:39035)
0927 10:50:10.482532 (+ 3356us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:10.482595 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:10.483766 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:10.484665 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:10.484674 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:10.485013 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:10.485656 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:10.485673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.486873 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.486876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:10.488274 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:10.488284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.489035 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.489039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:10.489233 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:10.489896 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:10.489916 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:10.492826 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:10.495161 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:10.495170 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:10.495200 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:10.495550 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:10.495887 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:10.495892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:10.495897 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:10.496062 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:10.496553 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:10.496561 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:10.496853 (+ 292us) client_negotiation.cc:769] Sending connection context
0927 10:50:10.497034 (+ 181us) client_negotiation.cc:240] Negotiation successful
0927 10:50:10.497405 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3108,"thread_start_us":139,"threads_started":1}
I20260927 10:50:10.500119 20818 heartbeater.cc:347] Connected to a master server at 127.9.39.190:41631
I20260927 10:50:10.500375 20818 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:10.500895 20818 heartbeater.cc:511] Master 127.9.39.190:41631 requested a full tablet report, sending...
I20260927 10:50:10.502439 20606 ts_manager.cc:194] Registered new tserver with Master: ccf47d667037407b829afa28baafa5ec (127.9.39.129:37355)
I20260927 10:50:10.503995 20606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:39035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:50:10.514629 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:45655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41631
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 10:50:10.634279 20826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:10.634625 20826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:10.634712 20826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:10.639254 20826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:10.639358 20826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:10.639500 20826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:50:10.644773 20826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.39.148:45655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:10.645754 20826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:10.646823 20826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:10.655948 20833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:10.656088 20832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:10.656322 20835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:10.656746 20826 server_base.cc:1034] running on GCE node
I20260927 10:50:10.657116 20826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:10.657732 20826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:10.658903 20826 hybrid_clock.cc:648] HybridClock initialized: now 1790506210658874 us; error 38 us; skew 500 ppm
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:10.661659 20826 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:50:10.662541 20826 webserver.cc:460] Webserver started at http://127.9.39.130:44867/ using document root <none> and password file <none>
I20260927 10:50:10.663105 20826 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:10.663199 20826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:10.663404 20826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:10.680321 20826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "e33ddb96d79e48b5ab96c705bda9786d"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "ad34c734b0a73be952ab225d3f5ab461"
server_key_iv: "904986dd720c357e468fdc735c5ceeec"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.681145 20826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "e33ddb96d79e48b5ab96c705bda9786d"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "ad34c734b0a73be952ab225d3f5ab461"
server_key_iv: "904986dd720c357e468fdc735c5ceeec"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.686347 20826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 10:50:10.707394 20842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:10.709151 20826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:50:10.709303 20826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "e33ddb96d79e48b5ab96c705bda9786d"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "ad34c734b0a73be952ab225d3f5ab461"
server_key_iv: "904986dd720c357e468fdc735c5ceeec"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.709437 20826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:10.738030 20826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:10.742048 20826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:10.742388 20826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:10.743131 20826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:10.744207 20826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:10.744299 20826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:10.744356 20826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:10.744387 20826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:10.755733 20826 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:45263
I20260927 10:50:10.755782 20955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:45263 every 8 connection(s)
I20260927 10:50:10.756954 20826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:50:10.762759 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 20826
I20260927 10:50:10.762876 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:50:10.763211 9374 external_mini_cluster.cc:1428] Setting key 871eed1e9a8d11c37881087715709e4b
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:50:10.776039 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:10.760681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:48903 (local address 127.9.39.190:41631)
0927 10:50:10.760862 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:10.760867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:10.761625 (+ 758us) server_negotiation.cc:407] Connection header received
0927 10:50:10.762544 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:10.762548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:10.762603 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:10.762723 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:10.763907 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.764631 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.766447 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.766924 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.767013 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:10.771644 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:10.771648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:10.771651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:10.771681 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:10.773527 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:10.774156 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:10.774161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:10.774163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:10.774269 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:10.774712 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:10.774718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:10.774722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:10.774940 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:10.775036 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:10.775650 (+ 614us) server_negotiation.cc:299] Negotiation successful
0927 10:50:10.775785 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
I20260927 10:50:10.777082 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:10.759540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:41631 (local address 127.9.39.130:48903)
0927 10:50:10.761470 (+ 1930us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:10.761511 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:10.762335 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:10.762909 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:10.762915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:10.763208 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:10.763757 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:10.763780 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.764794 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.764797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:10.766243 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:10.766257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:10.767220 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:10.767226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:10.767441 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:10.768968 (+ 1527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:10.769004 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:10.771448 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:10.773683 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:10.773689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:10.773705 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:10.774028 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:10.774393 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:10.774398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:10.774402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:10.774575 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:10.775088 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:10.775094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:10.775408 (+ 314us) client_negotiation.cc:769] Sending connection context
0927 10:50:10.775601 (+ 193us) client_negotiation.cc:240] Negotiation successful
0927 10:50:10.775952 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"thread_start_us":144,"threads_started":1}
I20260927 10:50:10.778510 20956 heartbeater.cc:347] Connected to a master server at 127.9.39.190:41631
I20260927 10:50:10.778757 20956 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:10.779361 20956 heartbeater.cc:511] Master 127.9.39.190:41631 requested a full tablet report, sending...
I20260927 10:50:10.780414 20606 ts_manager.cc:194] Registered new tserver with Master: e33ddb96d79e48b5ab96c705bda9786d (127.9.39.130:45263)
I20260927 10:50:10.781229 20606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:48903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:50:10.793769 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:45655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41631
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:50:10.939153 20961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:10.939528 20961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:10.939697 20961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:10.945598 20961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:10.945732 20961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:10.945874 20961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:50:10.952940 20961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.39.148:45655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:10.954442 20961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:10.955899 20961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:10.967703 20967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:10.968354 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:10.968799 20961 server_base.cc:1034] running on GCE node
W20260927 10:50:10.969008 20966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:10.969507 20961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:10.970449 20961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:10.971706 20961 hybrid_clock.cc:648] HybridClock initialized: now 1790506210971686 us; error 36 us; skew 500 ppm
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:10.974781 20961 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:50:10.975829 20961 webserver.cc:460] Webserver started at http://127.9.39.131:34045/ using document root <none> and password file <none>
I20260927 10:50:10.976377 20961 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:10.976467 20961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:10.976728 20961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:10 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:10.990656 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "4f85af4942054015a09153d1dd15749e"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "020f3418b3476b664956cf68412569f7"
server_key_iv: "d8b452dfe9448020494cce964a922aac"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.991600 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "4f85af4942054015a09153d1dd15749e"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "020f3418b3476b664956cf68412569f7"
server_key_iv: "d8b452dfe9448020494cce964a922aac"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:10.995656 20961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 10:50:11.030750 20976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:11.032780 20961 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.002s sys 0.000s
I20260927 10:50:11.032987 20961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "4f85af4942054015a09153d1dd15749e"
format_stamp: "Formatted at 2026-09-27 10:50:10 on dist-test-slave-p9q6"
server_key: "020f3418b3476b664956cf68412569f7"
server_key_iv: "d8b452dfe9448020494cce964a922aac"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:11.033210 20961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 27 10:50:11 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:11.070274 20961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:11.074009 20961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:11.074234 20961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:11.075948 20961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:11.077026 20961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:11.077105 20961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:11.077168 20961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:11.077219 20961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:11.089658 20961 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:35413
I20260927 10:50:11.091863 20961 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:50:11.092362 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 20961
I20260927 10:50:11.092473 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:50:11.092778 9374 external_mini_cluster.cc:1428] Setting key 28251e32996d414c637ce5426b0f43dd
I20260927 10:50:11.089685 21089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:35413 every 8 connection(s)
Sep 27 10:50:11 dist-test-slave-p9q6 krb5kdc[20130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506210, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:11.112142 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.098022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:33869 (local address 127.9.39.190:41631)
0927 10:50:11.098136 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.098140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.099116 (+ 976us) server_negotiation.cc:407] Connection header received
0927 10:50:11.100251 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.100254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.100301 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.100352 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:11.101847 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.102662 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.104255 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.104598 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.104672 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.108059 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:11.108063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:11.108066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:11.108089 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:11.109808 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:11.110405 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:11.110409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:11.110411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:11.110490 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:11.110922 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:11.110926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:11.110928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:11.111123 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:11.111212 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.111838 (+ 626us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.111972 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 10:50:11.113636 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.096983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:41631 (local address 127.9.39.131:33869)
0927 10:50:11.098979 (+ 1996us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.099032 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:11.100007 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:11.100748 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:11.100756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:11.101074 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:11.101687 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.101707 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.102840 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.102844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.104117 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.104129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.104856 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.104859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.105523 (+ 664us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.106227 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:11.106248 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:11.107889 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:11.109947 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:11.109952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:11.109968 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:11.110295 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:11.110591 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:11.110594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:11.110596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:11.110812 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:11.111293 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:11.111299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:11.111565 (+ 266us) client_negotiation.cc:769] Sending connection context
0927 10:50:11.112358 (+ 793us) client_negotiation.cc:240] Negotiation successful
0927 10:50:11.112649 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1790,"thread_start_us":155,"threads_started":1}
I20260927 10:50:11.114912 21090 heartbeater.cc:347] Connected to a master server at 127.9.39.190:41631
I20260927 10:50:11.115125 21090 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:11.115648 21090 heartbeater.cc:511] Master 127.9.39.190:41631 requested a full tablet report, sending...
I20260927 10:50:11.116609 20606 ts_manager.cc:194] Registered new tserver with Master: 4f85af4942054015a09153d1dd15749e (127.9.39.131:35413)
I20260927 10:50:11.117329 20606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:33869
I20260927 10:50:11.117909 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:50:11.132886 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.123099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43450 (local address 127.9.39.190:41631)
0927 10:50:11.123213 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.123217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.123795 (+ 578us) server_negotiation.cc:407] Connection header received
0927 10:50:11.123952 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.123955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.123993 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.124038 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:11.125038 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.125854 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.127161 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.127496 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.127562 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.129264 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:11.129268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:11.129271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:11.129293 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:11.131019 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:11.131555 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:11.131559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:11.131560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:11.131641 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:11.131961 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:11.131965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:11.131969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:11.132152 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:11.132260 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.132620 (+ 360us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.132736 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:50:11.136044 20605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:50:11.139135 20605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:50:11.169449 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.150672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:35413 (local address 127.0.0.1:57864)
0927 10:50:11.155083 (+ 4411us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.155127 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:11.155247 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:11.156024 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:11.156030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:11.156052 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:11.156165 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.156171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.158420 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.158423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.161713 (+ 3290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.161723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.168963 (+ 7240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.168966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.169070 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.169104 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 10:50:11.169200 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 10:50:11.169306 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4257,"thread_start_us":97,"threads_started":1}
I20260927 10:50:11.170265 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.150806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57864 (local address 127.9.39.131:35413)
0927 10:50:11.154858 (+ 4052us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.154869 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.155373 (+ 504us) server_negotiation.cc:407] Connection header received
0927 10:50:11.155453 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.155458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.155697 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.155857 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:11.156292 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.158295 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.161862 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.168655 (+ 6793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.168763 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.168911 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.170034 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.170130 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3949,"thread_start_us":125,"threads_started":1}
I20260927 10:50:11.177575 21012 tablet_service.cc:1467] Processing CreateTablet for tablet dbd93e0d1d8a4b2b92109e3a02892ebb (DEFAULT_TABLE table=test-table [id=d9c395dd350542c889375d01aee31ad5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:50:11.179015 21012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd93e0d1d8a4b2b92109e3a02892ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:11.189077 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.152522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45263 (local address 127.0.0.1:41472)
0927 10:50:11.157225 (+ 4703us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.157239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:11.157315 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:11.161091 (+ 3776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:11.161095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:11.161111 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:11.161211 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.161218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.165439 (+ 4221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.165442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.166654 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.166664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.187896 (+ 21232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.187899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.188013 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.188039 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 10:50:11.188112 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 10:50:11.188201 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4646,"thread_start_us":74,"threads_started":1}
I20260927 10:50:11.191123 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.153326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37355 (local address 127.0.0.1:38244)
0927 10:50:11.164117 (+ 10791us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.164131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:11.164209 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:11.167536 (+ 3327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:11.167539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:11.167555 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:11.167707 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.167713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.171852 (+ 4139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.171855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.172998 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.173008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.190721 (+ 17713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.190725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.190834 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.190860 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 10:50:11.190932 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 10:50:11.191002 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10735,"thread_start_us":68,"threads_started":1}
I20260927 10:50:11.191812 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.152876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.9.39.130:45263)
0927 10:50:11.160402 (+ 7526us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.160413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.160434 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:50:11.160518 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.160523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.160754 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.160915 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:11.163138 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.165292 (+ 2154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.173149 (+ 7857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.187536 (+ 14387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.187661 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.187822 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.191553 (+ 3731us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.191660 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7410,"thread_start_us":135,"threads_started":1}
I20260927 10:50:11.194911 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.153674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.9.39.129:37355)
0927 10:50:11.166949 (+ 13275us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.166956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.166978 (+ 22us) server_negotiation.cc:407] Connection header received
0927 10:50:11.167049 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.167053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.167263 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.167407 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:11.170512 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.171708 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.189562 (+ 17854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.190389 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.190497 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.190630 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.194645 (+ 4015us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.194775 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13170,"thread_start_us":58,"threads_started":1}
I20260927 10:50:11.194851 20881 tablet_service.cc:1467] Processing CreateTablet for tablet dbd93e0d1d8a4b2b92109e3a02892ebb (DEFAULT_TABLE table=test-table [id=d9c395dd350542c889375d01aee31ad5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:50:11.196014 20881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd93e0d1d8a4b2b92109e3a02892ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:11.197510 20752 tablet_service.cc:1467] Processing CreateTablet for tablet dbd93e0d1d8a4b2b92109e3a02892ebb (DEFAULT_TABLE table=test-table [id=d9c395dd350542c889375d01aee31ad5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:50:11.198412 20752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd93e0d1d8a4b2b92109e3a02892ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:11.205706 21110 tablet_bootstrap.cc:492] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Bootstrap starting.
I20260927 10:50:11.207908 21110 tablet_bootstrap.cc:654] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:11.208259 21111 tablet_bootstrap.cc:492] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec: Bootstrap starting.
I20260927 10:50:11.210242 21111 tablet_bootstrap.cc:654] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:11.211074 21109 tablet_bootstrap.cc:492] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Bootstrap starting.
I20260927 10:50:11.211576 21110 log.cc:824] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:11.212886 21111 log.cc:824] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:11.213534 21109 tablet_bootstrap.cc:654] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:11.214884 21110 tablet_bootstrap.cc:492] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: No bootstrap required, opened a new log
I20260927 10:50:11.215108 21110 ts_tablet_manager.cc:1397] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260927 10:50:11.216572 21109 log.cc:824] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:11.218941 21110 raft_consensus.cc:348] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.219328 21110 raft_consensus.cc:374] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:11.219424 21110 raft_consensus.cc:729] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33ddb96d79e48b5ab96c705bda9786d, State: Initialized, Role: FOLLOWER
I20260927 10:50:11.219923 21110 consensus_queue.cc:260] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.220928 21110 ts_tablet_manager.cc:1428] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:50:11.220999 21109 tablet_bootstrap.cc:492] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: No bootstrap required, opened a new log
I20260927 10:50:11.221163 21109 ts_tablet_manager.cc:1397] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260927 10:50:11.221292 20956 heartbeater.cc:503] Master 127.9.39.190:41631 was elected leader, sending a full tablet report...
I20260927 10:50:11.222781 21111 tablet_bootstrap.cc:492] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec: No bootstrap required, opened a new log
I20260927 10:50:11.223007 21111 ts_tablet_manager.cc:1397] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260927 10:50:11.225114 21109 raft_consensus.cc:348] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.227169 21111 raft_consensus.cc:348] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.227468 21111 raft_consensus.cc:374] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:11.227517 21111 raft_consensus.cc:729] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf47d667037407b829afa28baafa5ec, State: Initialized, Role: FOLLOWER
I20260927 10:50:11.227977 21111 consensus_queue.cc:260] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.228766 20818 heartbeater.cc:503] Master 127.9.39.190:41631 was elected leader, sending a full tablet report...
I20260927 10:50:11.228813 21111 ts_tablet_manager.cc:1428] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:50:11.230780 21109 raft_consensus.cc:374] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:11.230911 21109 raft_consensus.cc:729] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f85af4942054015a09153d1dd15749e, State: Initialized, Role: FOLLOWER
I20260927 10:50:11.231396 21109 consensus_queue.cc:260] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.232393 21109 ts_tablet_manager.cc:1428] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260927 10:50:11.232735 21090 heartbeater.cc:503] Master 127.9.39.190:41631 was elected leader, sending a full tablet report...
I20260927 10:50:11.247164 21116 raft_consensus.cc:482] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:50:11.247396 21116 raft_consensus.cc:504] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.249109 21116 leader_election.cc:288] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f85af4942054015a09153d1dd15749e (127.9.39.131:35413), e33ddb96d79e48b5ab96c705bda9786d (127.9.39.130:45263)
I20260927 10:50:11.253899 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.249595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:54677 (local address 127.9.39.130:45263)
0927 10:50:11.249738 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.249742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.249755 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:50:11.249800 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.249802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.249841 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.249916 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:11.250278 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.251056 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.252666 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.253235 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.253325 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.253375 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.253717 (+ 342us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.253784 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 10:50:11.254470 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.249297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45263 (local address 127.9.39.129:54677)
0927 10:50:11.249549 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.249566 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:11.249659 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:11.249959 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:11.249962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:11.249982 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:11.250123 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.250130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.251226 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.251229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.252529 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.252540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.253477 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.253481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.253572 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.253595 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 10:50:11.254224 (+ 629us) client_negotiation.cc:240] Negotiation successful
0927 10:50:11.254318 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":4,"thread_start_us":87,"threads_started":1}
I20260927 10:50:11.255332 20907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd93e0d1d8a4b2b92109e3a02892ebb" candidate_uuid: "ccf47d667037407b829afa28baafa5ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33ddb96d79e48b5ab96c705bda9786d" is_pre_election: true
I20260927 10:50:11.255416 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.248866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:60025 (local address 127.9.39.131:35413)
0927 10:50:11.248973 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.248977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.250077 (+ 1100us) server_negotiation.cc:407] Connection header received
0927 10:50:11.250236 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.250239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.250280 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.250321 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:11.250677 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.251429 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.254093 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.254675 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.254788 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.254831 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.255232 (+ 401us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.255303 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 10:50:11.255438 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.248795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:35413 (local address 127.9.39.129:60025)
0927 10:50:11.250006 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.250019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:11.250132 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:11.250430 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:11.250433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:11.250446 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:11.250557 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.250562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.251576 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.251579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.252818 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:11.252827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.255021 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.255024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:11.255120 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.255141 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 10:50:11.255212 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 10:50:11.255271 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":49,"thread_start_us":103,"threads_started":1}
I20260927 10:50:11.255604 20907 raft_consensus.cc:2454] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf47d667037407b829afa28baafa5ec in term 0.
I20260927 10:50:11.256078 20705 leader_election.cc:302] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccf47d667037407b829afa28baafa5ec, e33ddb96d79e48b5ab96c705bda9786d; no voters:
I20260927 10:50:11.256258 21028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd93e0d1d8a4b2b92109e3a02892ebb" candidate_uuid: "ccf47d667037407b829afa28baafa5ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f85af4942054015a09153d1dd15749e" is_pre_election: true
I20260927 10:50:11.256394 21116 raft_consensus.cc:2781] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:50:11.256493 21116 raft_consensus.cc:482] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:50:11.256531 21116 raft_consensus.cc:3037] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:11.256539 21028 raft_consensus.cc:2454] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf47d667037407b829afa28baafa5ec in term 0.
I20260927 10:50:11.257694 21116 raft_consensus.cc:504] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.258045 21116 leader_election.cc:288] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [CANDIDATE]: Term 1 election: Requested vote from peers 4f85af4942054015a09153d1dd15749e (127.9.39.131:35413), e33ddb96d79e48b5ab96c705bda9786d (127.9.39.130:45263)
I20260927 10:50:11.258332 21028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd93e0d1d8a4b2b92109e3a02892ebb" candidate_uuid: "ccf47d667037407b829afa28baafa5ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f85af4942054015a09153d1dd15749e"
I20260927 10:50:11.258464 20907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd93e0d1d8a4b2b92109e3a02892ebb" candidate_uuid: "ccf47d667037407b829afa28baafa5ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33ddb96d79e48b5ab96c705bda9786d"
I20260927 10:50:11.258466 21028 raft_consensus.cc:3037] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:11.258559 20907 raft_consensus.cc:3037] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:11.259656 21028 raft_consensus.cc:2454] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf47d667037407b829afa28baafa5ec in term 1.
I20260927 10:50:11.259656 20907 raft_consensus.cc:2454] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf47d667037407b829afa28baafa5ec in term 1.
W20260927 10:50:11.259925 20957 tablet.cc:2369] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:50:11.260004 20708 leader_election.cc:302] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f85af4942054015a09153d1dd15749e, ccf47d667037407b829afa28baafa5ec; no voters:
I20260927 10:50:11.260738 21116 raft_consensus.cc:2781] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:50:11.260990 21116 raft_consensus.cc:686] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [term 1 LEADER]: Becoming Leader. State: Replica: ccf47d667037407b829afa28baafa5ec, State: Running, Role: LEADER
I20260927 10:50:11.261315 21116 consensus_queue.cc:237] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } }
I20260927 10:50:11.264382 20606 catalog_manager.cc:5495] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf47d667037407b829afa28baafa5ec (127.9.39.129). New cstate: current_term: 1 leader_uuid: "ccf47d667037407b829afa28baafa5ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf47d667037407b829afa28baafa5ec" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 } health_report { overall_health: UNKNOWN } } }
I20260927 10:50:11.280260 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.276916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.9.39.129:37355)
0927 10:50:11.277040 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:11.277044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:11.277056 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:50:11.277117 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:11.277119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:11.277163 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:11.277270 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:50:11.277493 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.277971 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.278873 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:11.279140 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:11.279252 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:11.279536 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:50:11.279914 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:50:11.280035 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:11.280089 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 10:50:11.280136 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 10:50:11.285547 20907 raft_consensus.cc:1261] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 1 FOLLOWER]: Refusing update from remote peer ccf47d667037407b829afa28baafa5ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:50:11.285547 21028 raft_consensus.cc:1261] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 1 FOLLOWER]: Refusing update from remote peer ccf47d667037407b829afa28baafa5ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:50:11.286229 21120 consensus_queue.cc:1035] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f85af4942054015a09153d1dd15749e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 35413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:50:11.286629 21120 consensus_queue.cc:1035] T dbd93e0d1d8a4b2b92109e3a02892ebb P ccf47d667037407b829afa28baafa5ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "e33ddb96d79e48b5ab96c705bda9786d" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:50:11.299300 21125 mvcc.cc:205] Tried to move back new op lower bound from 7333913441418833920 to 7333913441329205248. Current Snapshot: MvccSnapshot[applied={T|T < 7333913441418833920}]
I20260927 10:50:11.299606 21124 mvcc.cc:205] Tried to move back new op lower bound from 7333913441418833920 to 7333913441329205248. Current Snapshot: MvccSnapshot[applied={T|T < 7333913441418833920}]
I20260927 10:50:11.309741 20605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:50:11.309898 20605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:50:11.312880 20605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d72e2241f99c47f4b8c91d498064cc2a: Sending DeleteTablet for 3 replicas of tablet dbd93e0d1d8a4b2b92109e3a02892ebb
I20260927 10:50:11.314226 21012 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd93e0d1d8a4b2b92109e3a02892ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:50:11 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:57864
I20260927 10:50:11.314713 20752 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd93e0d1d8a4b2b92109e3a02892ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:50:11 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260927 10:50:11.315140 20881 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd93e0d1d8a4b2b92109e3a02892ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:50:11 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:41472
I20260927 10:50:11.316504 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 20687
I20260927 10:50:11.318439 21134 tablet_replica.cc:330] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: stopping tablet replica
I20260927 10:50:11.318828 21134 raft_consensus.cc:2229] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:50:11.324505 21132 tablet_replica.cc:330] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: stopping tablet replica
I20260927 10:50:11.324934 21134 raft_consensus.cc:2258] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:50:11.325361 21132 raft_consensus.cc:2229] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:50:11.325922 21132 raft_consensus.cc:2258] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:50:11.327695 21132 ts_tablet_manager.cc:1905] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:50:11.327831 21134 ts_tablet_manager.cc:1905] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:50:11.330652 21134 ts_tablet_manager.cc:1918] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:50:11.330850 21134 log.cc:1196] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/dbd93e0d1d8a4b2b92109e3a02892ebb
I20260927 10:50:11.331075 21132 ts_tablet_manager.cc:1918] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:50:11.331259 21134 ts_tablet_manager.cc:1939] T dbd93e0d1d8a4b2b92109e3a02892ebb P e33ddb96d79e48b5ab96c705bda9786d: Deleting consensus metadata
I20260927 10:50:11.332511 20590 catalog_manager.cc:4841] TS e33ddb96d79e48b5ab96c705bda9786d (127.9.39.130:45263): tablet dbd93e0d1d8a4b2b92109e3a02892ebb (table test-table [id=d9c395dd350542c889375d01aee31ad5]) successfully deleted
W20260927 10:50:11.333243 20593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:37355 (error 108)
I20260927 10:50:11.333868 21132 log.cc:1196] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/dbd93e0d1d8a4b2b92109e3a02892ebb
I20260927 10:50:11.334263 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:11.333992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37355 (local address 127.0.0.1:38266)
0927 10:50:11.334148 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:11.334200 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:37355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260927 10:50:11.334263 21132 ts_tablet_manager.cc:1939] T dbd93e0d1d8a4b2b92109e3a02892ebb P 4f85af4942054015a09153d1dd15749e: Deleting consensus metadata
W20260927 10:50:11.334434 20593 catalog_manager.cc:4584] TS ccf47d667037407b829afa28baafa5ec (127.9.39.129:37355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbd93e0d1d8a4b2b92109e3a02892ebb: Network error: Client connection negotiation failed: client connection to 127.9.39.129:37355: connect: Connection refused (error 111)
I20260927 10:50:11.335103 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 20826
I20260927 10:50:11.338889 20593 catalog_manager.cc:4841] TS 4f85af4942054015a09153d1dd15749e (127.9.39.131:35413): tablet dbd93e0d1d8a4b2b92109e3a02892ebb (table test-table [id=d9c395dd350542c889375d01aee31ad5]) successfully deleted
I20260927 10:50:11.342315 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 20961
I20260927 10:50:11.348249 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 20574
I20260927 10:50:11.354332 9374 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 10:50:11.688937 9374 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 10:50:11.689070 9374 mini_ranger.cc:66] Stopping Ranger...
I20260927 10:50:12.034897 9374 mini_ranger.cc:68] Stopped Ranger
2026-09-27 10:50:12.035 UTC [20160] LOG: received smart shutdown request
2026-09-27 10:50:12.038 UTC [20160] LOG: background worker "logical replication launcher" (PID 20169) exited with exit code 1
2026-09-27 10:50:12.038 UTC [20164] LOG: shutting down
2026-09-27 10:50:12.184 UTC [20160] LOG: database system is shut down
2026-09-27T10:50:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45459 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:50:12 dist-test-slave-p9q6 krb5kdc[21145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:50:12 dist-test-slave-p9q6 krb5kdc[21145](info): set up 2 sockets
Sep 27 10:50:12 dist-test-slave-p9q6 krb5kdc[21145](info): commencing operation
krb5kdc: starting...
W20260927 10:50:13.322568 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:50:13 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:50:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/rangeradmin_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/rangeradmin_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/rangerlookup_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/rangerlookup_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
I20260927 10:50:13.479135 9374 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasku3mICI/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/postgres -l logfile start
2026-09-27 10:50:15.851 UTC [21175] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 10:50:15.851 UTC [21175] LOG: listening on IPv4 address "127.9.39.148", port 48579
2026-09-27 10:50:15.854 UTC [21175] LOG: listening on Unix socket "/tmp/.s.PGSQL.48579"
2026-09-27 10:50:15.875 UTC [21178] LOG: database system was shut down at 2026-09-27 10:50:14 UTC
2026-09-27 10:50:15.878 UTC [21175] LOG: database system is ready to accept connections
I20260927 10:50:16.830674 9374 mini_postgres.cc:96] Postgres bound to 48579
2026-09-27 10:50:16.836 UTC [21186] FATAL: database "slave" does not exist
127.9.39.148:48579 - accepting connections
I20260927 10:50:16.837225 9374 mini_ranger.cc:161] Starting Ranger...
I20260927 10:50:16.851516 9374 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 10:50:17.280109 9374 mini_ranger.cc:87] Created ranger Postgres database
I20260927 10:50:17.280196 9374 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-admin
2026-09-27 10:50:17,547 [I] DB FLAVOR :POSTGRES
2026-09-27 10:50:17,548 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 10:50:17,548 [I] Checking connection..
2026-09-27 10:50:17,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 10:50:17,795 [I] Checking connection passed.
2026-09-27 10:50:17,795 [I] --------- Verifying version history table ---------
2026-09-27 10:50:17,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 10:50:18,025 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 10:50:18,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 10:50:18,272 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 10:50:18,272 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 10:50:18,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 10:50:18.515 UTC [21250] WARNING: there is no transaction in progress
2026-09-27 10:50:18,521 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 10:50:18,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 10:50:18,754 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 10:50:18,754 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 10:50:18,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 10:50:19,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 10:50:19,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p9q6.c.gcp-upstream.internal','N') ;"
2026-09-27 10:50:19,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 10:50:19,718 [I] Table x_portal_user does not exist in database ranger
2026-09-27 10:50:19,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 10:50:19,947 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 10:50:19,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 10:50:20,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 10:50:20,443 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 10:50:20,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 10:50:21.377 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.392 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.407 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.421 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.436 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.452 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.503 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.509 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.523 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.532 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.542 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.555 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.565 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.571 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.578 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:21.585 UTC [21358] WARNING: there is no transaction in progress
2026-09-27 10:50:22,200 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 10:50:22,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 10:50:22,418 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 10:50:22,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 10:50:22,636 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 10:50:22,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 10:50:22,885 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 10:50:22,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 10:50:23,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 10:50:23,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p9q6.c.gcp-upstream.internal';"
2026-09-27 10:50:23,586 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 10:50:23,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 10:50:23,801 [I] DB_PATCHES have already been applied
I20260927 10:50:23.807149 9374 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-admin:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasku3mICI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 10:50:23.807210 9374 mini_ranger.cc:193] Using host: 127.9.39.148
I20260927 10:50:23.810417 9374 mini_ranger.cc:227] Ranger admin URL: http://127.9.39.148:40223
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 10:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 10:50:24.887792 9374 mini_ranger.cc:160] Time spent starting Ranger: real 8.051s user 0.000s sys 0.009s
Sep 27, 2026 10:50:45 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 10:50:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 10:50:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 10:50:45 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 10:50:46 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 10:50:46.620561 9374 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.39.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.39.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/rangerkms_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.39.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/rangerkms_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.39.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.39.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.39.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/HTTP_127.9.39.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 10:50:46 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506246, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 10:50:46.704602 9374 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 10:50:46.713747 9374 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 10:50:47.601619 9374 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 10:50:47.601706 9374 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms
I20260927 10:50:47.601729 9374 mini_ranger_kms.cc:227] Using postgres at 127.9.39.148:48579
2026-09-27 10:50:47,771 [I] DB FLAVOR :POSTGRES
2026-09-27 10:50:47,771 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 10:50:47,771 [I] Checking connection
2026-09-27 10:50:47,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 10:50:48,014 [I] connection success
2026-09-27 10:50:48,015 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 10:50:48,015 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 10:50:48,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 10:50:48,267 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 10:50:48,267 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 10:50:48,268 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 10:50:48,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.39.148:48579/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 10:50:48,541 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 10:50:48.987879 9374 mini_ranger_kms.cc:310] Created kms service
I20260927 10:50:49.152133 9374 mini_ranger_kms.cc:326] Created kudu user
I20260927 10:50:49.195288 9374 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 10:50:49 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506246, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:49.613641 9374 mini_ranger_kms.cc:384] Added ranger policy
I20260927 10:50:49.613814 9374 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms:/tmp/dist-test-tasku3mICI/build/debug/bin/postgresql.jar:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasku3mICI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 10:50:49.613837 9374 mini_ranger_kms.cc:240] Using host: 127.9.39.148
I20260927 10:50:49.617480 9374 mini_ranger_kms.cc:276] Ranger KMS PID: 21538
I20260927 10:50:49.617568 9374 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.39.148:39847
Sep 27, 2026 10:50:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 10:50:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 10:50:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 10:50:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 10:50:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 10:50:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 10:50:51.857856 9374 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.153s user 0.000s sys 0.153s
I20260927 10:50:51.857969 9374 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 10:50:51.858031 9374 mini_ranger_kms.cc:401] 127.9.39.148:39847/kms/v1/keys
Sep 27 10:50:51 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790506251, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.39.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 10:50:52 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506251, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.39.148@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
Sep 27 10:50:52 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506251, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.39.148@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
Sep 27 10:50:54 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:50:54.801904 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40743
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40743
--ranger_config_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:50:54.910202 21589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:54.910512 21589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:54.910588 21589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:54.914835 21589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:50:54.914927 21589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:54.914963 21589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:54.915019 21589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:50:54.915053 21589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:50:54.920415 21589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.39.148:39847/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40743
--ranger_config_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:54.921412 21589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:54.922384 21589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:54.930807 21595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:54.930877 21594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:54.930914 21597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:54.931012 21589 server_base.cc:1034] running on GCE node
I20260927 10:50:54.931535 21589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:54.932545 21589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:54.933729 21589 hybrid_clock.cc:648] HybridClock initialized: now 1790506254933704 us; error 48 us; skew 500 ppm
Sep 27 10:50:54 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:54.936468 21589 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:50:54.937316 21589 webserver.cc:460] Webserver started at http://127.9.39.190:34365/ using document root <none> and password file <none>
I20260927 10:50:54.937831 21589 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:54.937906 21589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:54.938117 21589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:54 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:54.999048 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "b0ac31535a484bff8267b6a55c91615d"
format_stamp: "Formatted at 2026-09-27 10:50:54 on dist-test-slave-p9q6"
server_key: "74c21838db7686ff13aa476ef065beca"
server_key_iv: "cd06a62f703d773ae772411878cc2fe7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:54.999639 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "b0ac31535a484bff8267b6a55c91615d"
format_stamp: "Formatted at 2026-09-27 10:50:54 on dist-test-slave-p9q6"
server_key: "74c21838db7686ff13aa476ef065beca"
server_key_iv: "cd06a62f703d773ae772411878cc2fe7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:55.003679 21589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 10:50:55.020646 21604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:55.022009 21589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:50:55.022152 21589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "b0ac31535a484bff8267b6a55c91615d"
format_stamp: "Formatted at 2026-09-27 10:50:54 on dist-test-slave-p9q6"
server_key: "74c21838db7686ff13aa476ef065beca"
server_key_iv: "cd06a62f703d773ae772411878cc2fe7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:55.022281 21589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:55.040262 21589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:55.043154 21589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:55.043337 21589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:55.051339 21589 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:40743
I20260927 10:50:55.051354 21656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:40743 every 8 connection(s)
I20260927 10:50:55.052282 21589 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:50:55.055063 21657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:55.059000 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 21589
I20260927 10:50:55.059126 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:50:55.059427 9374 external_mini_cluster.cc:1428] Setting key 5ee83212f15cacd539806d44da4f94e0
I20260927 10:50:55.060793 21657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Bootstrap starting.
I20260927 10:50:55.063472 21657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:55.064361 21657 log.cc:824] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:55.066263 21657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: No bootstrap required, opened a new log
Sep 27 10:50:55 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:55.070583 21657 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:55.070904 21657 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:55.070990 21657 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ac31535a484bff8267b6a55c91615d, State: Initialized, Role: FOLLOWER
I20260927 10:50:55.071525 21657 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:55.071724 21657 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:50:55.071810 21657 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:50:55.071913 21657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:55.073020 21657 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:55.073376 21657 leader_election.cc:302] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0ac31535a484bff8267b6a55c91615d; no voters:
I20260927 10:50:55.073704 21657 leader_election.cc:288] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:50:55.073840 21662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:50:55.074137 21662 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 LEADER]: Becoming Leader. State: Replica: b0ac31535a484bff8267b6a55c91615d, State: Running, Role: LEADER
I20260927 10:50:55.074486 21662 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:55.075088 21657 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:50:55.075795 21662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0ac31535a484bff8267b6a55c91615d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } } }
I20260927 10:50:55.075987 21662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: This master's current role is: LEADER
I20260927 10:50:55.076229 21662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0ac31535a484bff8267b6a55c91615d. Latest consensus state: current_term: 1 leader_uuid: "b0ac31535a484bff8267b6a55c91615d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } } }
I20260927 10:50:55.076287 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:55.060880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.9.39.190:40743)
0927 10:50:55.061300 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:55.061318 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:55.061358 (+ 40us) server_negotiation.cc:407] Connection header received
0927 10:50:55.062188 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:55.062210 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:55.062616 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:55.062982 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:55.063874 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:55.065178 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:55.066433 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:55.067201 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:55.067359 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:55.070448 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:55.070453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:55.070468 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:55.070497 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:55.072543 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:55.073059 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:55.073065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:55.073070 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:55.073195 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:55.073480 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:55.073483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:55.073485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:55.074020 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:55.074885 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:55.075028 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 10:50:55.075301 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260927 10:50:55.076370 21662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: This master's current role is: LEADER
I20260927 10:50:55.077308 21657 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 10:50:55.081758 21671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:50:55.086833 21671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:50:55.095378 21671 catalog_manager.cc:1347] Generated new cluster ID: 14e28561c1e742c39a9c324a47415a4e
I20260927 10:50:55.095441 21671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:50:55.116569 21671 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:50:55.118021 21671 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:50:55.123445 21671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Generated new TSK 0
I20260927 10:50:55.125001 21671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:50:55.805594 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40743
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:50:55.921502 21702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:55.921778 21702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:55.921864 21702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:55.926226 21702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:55.926436 21702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:55.926595 21702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:50:55.934376 21702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.39.148:39847/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:55.935536 21702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:55.936561 21702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:55.946389 21708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:55.946448 21707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:55.946586 21710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:55.947083 21702 server_base.cc:1034] running on GCE node
I20260927 10:50:55.947480 21702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:55.948120 21702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:55.949291 21702 hybrid_clock.cc:648] HybridClock initialized: now 1790506255949258 us; error 47 us; skew 500 ppm
Sep 27 10:50:55 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506255, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:55.952136 21702 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:50:55.953161 21702 webserver.cc:460] Webserver started at http://127.9.39.129:40913/ using document root <none> and password file <none>
I20260927 10:50:55.953698 21702 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:55.953758 21702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:55.953954 21702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:55 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506255, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:55.972839 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "a28fdcb17195400997150c29aa3c4f44"
format_stamp: "Formatted at 2026-09-27 10:50:55 on dist-test-slave-p9q6"
server_key: "6f30459348898d9412f487fe233307cc"
server_key_iv: "37f8ba4ae6e71cd8cac7aa2687b1f2f2"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:55.973701 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "a28fdcb17195400997150c29aa3c4f44"
format_stamp: "Formatted at 2026-09-27 10:50:55 on dist-test-slave-p9q6"
server_key: "6f30459348898d9412f487fe233307cc"
server_key_iv: "37f8ba4ae6e71cd8cac7aa2687b1f2f2"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:55.979032 21702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 10:50:55.999089 21719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.000748 21702 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260927 10:50:56.000931 21702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "a28fdcb17195400997150c29aa3c4f44"
format_stamp: "Formatted at 2026-09-27 10:50:55 on dist-test-slave-p9q6"
server_key: "6f30459348898d9412f487fe233307cc"
server_key_iv: "37f8ba4ae6e71cd8cac7aa2687b1f2f2"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.001144 21702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:56.022212 21702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:56.025976 21702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:56.026173 21702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:56.026873 21702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:56.027930 21702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:56.028051 21702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.028175 21702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:56.028256 21702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.039825 21702 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:37999
I20260927 10:50:56.041066 21702 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:50:56.045619 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 21702
I20260927 10:50:56.045748 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:50:56.046059 9374 external_mini_cluster.cc:1428] Setting key 451a6fb962a3a7be38deadd409192de6
I20260927 10:50:56.047298 21832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:37999 every 8 connection(s)
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506255, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:56.083164 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:56.050625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:34671 (local address 127.9.39.190:40743)
0927 10:50:56.057470 (+ 6845us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:56.057480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:56.064509 (+ 7029us) server_negotiation.cc:407] Connection header received
0927 10:50:56.069969 (+ 5460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:56.069973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:56.070037 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:56.070116 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:56.071942 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.072805 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.074607 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.075016 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.075125 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:56.078408 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:56.078412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:56.078415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:56.078446 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:56.080325 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:56.081054 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:56.081060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:56.081063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:56.081162 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:56.081635 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:56.081640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:56.081643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:56.081896 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:56.081977 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:56.082835 (+ 858us) server_negotiation.cc:299] Negotiation successful
0927 10:50:56.082977 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6754,"thread_start_us":122,"threads_started":1}
I20260927 10:50:56.084424 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:56.049541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40743 (local address 127.9.39.129:34671)
0927 10:50:56.064295 (+ 14754us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:56.064385 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:56.067801 (+ 3416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:56.070632 (+ 2831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:56.070642 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:56.071052 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:56.071765 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:56.071786 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.072980 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.072984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:56.074456 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:56.074470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.075221 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.075228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:56.075463 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:56.076256 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:56.076282 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:56.078173 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:56.080460 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:56.080470 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:56.080492 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:56.080915 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:56.081288 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:56.081294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:56.081297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:56.081512 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:56.082069 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:56.082080 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:56.082501 (+ 421us) client_negotiation.cc:769] Sending connection context
0927 10:50:56.082779 (+ 278us) client_negotiation.cc:240] Negotiation successful
0927 10:50:56.083154 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14476,"thread_start_us":144,"threads_started":1}
I20260927 10:50:56.086006 21833 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40743
I20260927 10:50:56.086279 21833 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:56.086905 21833 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:50:56.088829 21621 ts_manager.cc:194] Registered new tserver with Master: a28fdcb17195400997150c29aa3c4f44 (127.9.39.129:37999)
I20260927 10:50:56.090368 21621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:34671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:50:56.127123 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40743
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 10:50:56.323168 21841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:56.323599 21841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:56.323714 21841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:56.330241 21841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:56.330399 21841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:56.330546 21841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:50:56.338419 21841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.39.148:39847/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:56.340392 21841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:56.341763 21841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:56.358354 21848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:56.359027 21850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:56.359360 21847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:56.364023 21841 server_base.cc:1034] running on GCE node
I20260927 10:50:56.364543 21841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:56.365355 21841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:56.366601 21841 hybrid_clock.cc:648] HybridClock initialized: now 1790506256366535 us; error 83 us; skew 500 ppm
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:56.374306 21841 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:50:56.375440 21841 webserver.cc:460] Webserver started at http://127.9.39.130:43685/ using document root <none> and password file <none>
I20260927 10:50:56.376124 21841 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:56.376195 21841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:56.376423 21841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:56.413827 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "34e5da7f38284565b506a87df496c400"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f5c0e93fd0560272651ba12c7d1891b2"
server_key_iv: "c79ec15a879c9cc020eaca45621ca44f"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.414645 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "34e5da7f38284565b506a87df496c400"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f5c0e93fd0560272651ba12c7d1891b2"
server_key_iv: "c79ec15a879c9cc020eaca45621ca44f"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.420176 21841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 10:50:56.451083 21857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.452749 21841 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260927 10:50:56.452899 21841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "34e5da7f38284565b506a87df496c400"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f5c0e93fd0560272651ba12c7d1891b2"
server_key_iv: "c79ec15a879c9cc020eaca45621ca44f"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.453047 21841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:56.483381 21841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:56.487306 21841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:56.487565 21841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:56.488369 21841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:56.489459 21841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:56.489575 21841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.489674 21841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:56.489738 21841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.500499 21841 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:45367
I20260927 10:50:56.500592 21970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:45367 every 8 connection(s)
I20260927 10:50:56.501739 21841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:50:56.511269 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 21841
I20260927 10:50:56.511404 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:50:56.511688 9374 external_mini_cluster.cc:1428] Setting key dfeac315fa7c28584f318b065732bb98
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:56.522598 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:56.503679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:45591 (local address 127.9.39.190:40743)
0927 10:50:56.503815 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:56.503819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:56.505474 (+ 1655us) server_negotiation.cc:407] Connection header received
0927 10:50:56.506601 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:56.506604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:56.506659 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:56.506776 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:56.508280 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.509135 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.510763 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.511110 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.511224 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:56.517247 (+ 6023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:56.517251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:56.517255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:56.517285 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:56.519152 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:56.519818 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:56.519821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:56.519823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:56.519908 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:56.520320 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:56.520336 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:56.520338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:56.520573 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:56.520663 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:56.522295 (+ 1632us) server_negotiation.cc:299] Negotiation successful
0927 10:50:56.522437 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:50:56.523721 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:56.504168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40743 (local address 127.9.39.130:45591)
0927 10:50:56.505317 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:56.505372 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:56.506382 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:56.507168 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:56.507177 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:56.507510 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:56.508137 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:56.508154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.509258 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.509262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:56.510619 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:56.510629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.511425 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.511429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:56.511639 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:56.512350 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:56.512370 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:56.517016 (+ 4646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:56.519299 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:56.519307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:56.519338 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:56.519699 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:56.520008 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:56.520012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:56.520014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:56.520213 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:56.520717 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:56.520724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:56.521076 (+ 352us) client_negotiation.cc:769] Sending connection context
0927 10:50:56.521286 (+ 210us) client_negotiation.cc:240] Negotiation successful
0927 10:50:56.521644 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"thread_start_us":118,"threads_started":1}
I20260927 10:50:56.525269 21971 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40743
I20260927 10:50:56.525516 21971 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:56.526204 21971 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:56.527311 21621 ts_manager.cc:194] Registered new tserver with Master: 34e5da7f38284565b506a87df496c400 (127.9.39.130:45367)
I20260927 10:50:56.527952 21621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:45591
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:50:56.568266 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40743
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
W20260927 10:50:56.728818 21976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:56.729245 21976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:56.729354 21976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:56.735594 21976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:56.735678 21976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:56.735765 21976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:50:56.743557 21976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.39.148:39847/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:56.745010 21976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:56.746345 21976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:56.758116 21984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:56.758209 21981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:56.758114 21982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:56.759200 21976 server_base.cc:1034] running on GCE node
I20260927 10:50:56.759655 21976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:56.760392 21976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:56.761660 21976 hybrid_clock.cc:648] HybridClock initialized: now 1790506256761567 us; error 104 us; skew 500 ppm
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:56.765129 21976 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:50:56.766125 21976 webserver.cc:460] Webserver started at http://127.9.39.131:39229/ using document root <none> and password file <none>
I20260927 10:50:56.766860 21976 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:56.766942 21976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:50:56.767211 21976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:56.794983 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "2276691b0c294d3cb8596ad06b2d38e3"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f72a63d45572401bff9c7fecc7b72c7c"
server_key_iv: "1847516e64e4f356f1d87076e4f4a24c"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.795745 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "2276691b0c294d3cb8596ad06b2d38e3"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f72a63d45572401bff9c7fecc7b72c7c"
server_key_iv: "1847516e64e4f356f1d87076e4f4a24c"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.801110 21976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260927 10:50:56.831048 21991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.832932 21976 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.002s sys 0.000s
I20260927 10:50:56.833145 21976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "2276691b0c294d3cb8596ad06b2d38e3"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f72a63d45572401bff9c7fecc7b72c7c"
server_key_iv: "1847516e64e4f356f1d87076e4f4a24c"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:56.833410 21976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:56.870795 21976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:56.874554 21976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:56.874790 21976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:56.875609 21976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:56.876775 21976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:56.876838 21976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.877002 21976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:56.877041 21976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:56.887816 21976 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:41085
I20260927 10:50:56.887873 22104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:41085 every 8 connection(s)
I20260927 10:50:56.889041 21976 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:50:56.894227 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 21976
I20260927 10:50:56.894378 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:50:56.894676 9374 external_mini_cluster.cc:1428] Setting key dd0049fe7f586a31d5b655c6ed9d0656
Sep 27 10:50:56 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:56.908474 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:56.892770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:45601 (local address 127.9.39.190:40743)
0927 10:50:56.892886 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:56.892891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:56.892909 (+ 18us) server_negotiation.cc:407] Connection header received
0927 10:50:56.893738 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:56.893742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:56.893802 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:56.893903 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:56.897898 (+ 3995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.898677 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.900444 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.900795 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.900877 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:56.903848 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:56.903852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:56.903855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:56.903891 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:56.905592 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:56.906188 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:56.906191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:56.906193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:56.906283 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:56.906744 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:56.906748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:56.906750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:56.906994 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:56.907098 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:56.908170 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0927 10:50:56.908313 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:50:56.910147 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:56.891641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40743 (local address 127.9.39.131:45601)
0927 10:50:56.892224 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:56.892281 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:56.893451 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:56.896648 (+ 3197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:56.896661 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:56.897032 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:56.897731 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:56.897753 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.898840 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.898843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:56.900293 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:56.900304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:56.901092 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:56.901096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:56.901289 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:56.901969 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:56.901989 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:56.903675 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:56.905705 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:56.905711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:56.905725 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:56.906089 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:56.906427 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:56.906431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:56.906433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:56.906630 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:56.907313 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:56.907320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:56.907835 (+ 515us) client_negotiation.cc:769] Sending connection context
0927 10:50:56.908751 (+ 916us) client_negotiation.cc:240] Negotiation successful
0927 10:50:56.909098 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":167,"threads_started":1}
I20260927 10:50:56.911303 22105 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40743
I20260927 10:50:56.911571 22105 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:56.912220 22105 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:56.913404 21621 ts_manager.cc:194] Registered new tserver with Master: 2276691b0c294d3cb8596ad06b2d38e3 (127.9.39.131:41085)
I20260927 10:50:56.914182 21621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:45601
I20260927 10:50:56.920288 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:50:56.921172 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 21702
I20260927 10:50:56.932772 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 21841
I20260927 10:50:56.940701 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 21976
I20260927 10:50:56.948318 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 21589
I20260927 10:50:56.955837 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40743
--webserver_interface=127.9.39.190
--webserver_port=34365
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40743
--ranger_config_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:50:57.064180 22108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:57.064463 22108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:57.064529 22108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:57.068475 22108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:50:57.068552 22108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:57.068588 22108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:57.068642 22108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:50:57.068681 22108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:50:57.073844 22108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40743
--ranger_config_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=34365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:57.074909 22108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:57.075893 22108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:57.084035 22116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.084208 22108 server_base.cc:1034] running on GCE node
W20260927 10:50:57.084031 22113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:57.084077 22114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.084793 22108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:57.085805 22108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:57.086963 22108 hybrid_clock.cc:648] HybridClock initialized: now 1790506257086946 us; error 28 us; skew 500 ppm
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:57.089525 22108 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:50:57.090371 22108 webserver.cc:460] Webserver started at http://127.9.39.190:34365/ using document root <none> and password file <none>
I20260927 10:50:57.090984 22108 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:57.091071 22108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:57.154590 22108 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260927 10:50:57.157217 22124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.158454 22108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:50:57.158587 22108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "b0ac31535a484bff8267b6a55c91615d"
format_stamp: "Formatted at 2026-09-27 10:50:54 on dist-test-slave-p9q6"
server_key: "74c21838db7686ff13aa476ef065beca"
server_key_iv: "cd06a62f703d773ae772411878cc2fe7"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:57.159230 22108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:57.174098 22108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:57.176612 22108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:57.176815 22108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:57.183820 22108 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:40743
I20260927 10:50:57.183842 22176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:40743 every 8 connection(s)
I20260927 10:50:57.184865 22108 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:50:57.188962 22177 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:50:57.190575 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Bootstrap starting.
I20260927 10:50:57.192431 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22108
I20260927 10:50:57.192863 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:37999
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:40743
--webserver_port=40913
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:50:57.206561 22177 log.cc:824] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:57.213177 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:50:57.213572 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Bootstrap complete.
I20260927 10:50:57.217644 22177 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:57.218042 22177 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ac31535a484bff8267b6a55c91615d, State: Initialized, Role: FOLLOWER
I20260927 10:50:57.218541 22177 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:57.218679 22177 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:50:57.218770 22177 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:50:57.218887 22177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 1 FOLLOWER]: Advancing to term 2
I20260927 10:50:57.220732 22177 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:57.221073 22177 leader_election.cc:302] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0ac31535a484bff8267b6a55c91615d; no voters:
I20260927 10:50:57.221387 22177 leader_election.cc:288] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 10:50:57.221483 22181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 2 FOLLOWER]: Leader election won for term 2
I20260927 10:50:57.221698 22181 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [term 2 LEADER]: Becoming Leader. State: Replica: b0ac31535a484bff8267b6a55c91615d, State: Running, Role: LEADER
I20260927 10:50:57.222060 22181 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } }
I20260927 10:50:57.222630 22177 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:50:57.223407 22182 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b0ac31535a484bff8267b6a55c91615d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } } }
I20260927 10:50:57.223551 22182 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: This master's current role is: LEADER
I20260927 10:50:57.224190 22183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0ac31535a484bff8267b6a55c91615d. Latest consensus state: current_term: 2 leader_uuid: "b0ac31535a484bff8267b6a55c91615d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ac31535a484bff8267b6a55c91615d" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40743 } } }
I20260927 10:50:57.224282 22183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d [sys.catalog]: This master's current role is: LEADER
I20260927 10:50:57.224445 22189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:50:57.224975 22177 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 10:50:57.231374 22189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:50:57.232231 22189 catalog_manager.cc:1259] Loaded cluster ID: 14e28561c1e742c39a9c324a47415a4e
I20260927 10:50:57.232295 22189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:50:57.235423 22189 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:50:57.236068 22189 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Loaded TSK: 0
I20260927 10:50:57.237435 22189 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 10:50:57.339586 22179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:57.339905 22179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:57.339980 22179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:57.346328 22179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:57.346443 22179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:57.346567 22179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:50:57.354373 22179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:37999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=40913
--tserver_master_addrs=127.9.39.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:57.355688 22179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:57.356964 22179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:57.369485 22215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:57.369874 22216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.370154 22179 server_base.cc:1034] running on GCE node
W20260927 10:50:57.370656 22218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.371140 22179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:57.371878 22179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:57.373059 22179 hybrid_clock.cc:648] HybridClock initialized: now 1790506257373039 us; error 49 us; skew 500 ppm
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:57.379156 22179 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:50:57.380276 22179 webserver.cc:460] Webserver started at http://127.9.39.129:40913/ using document root <none> and password file <none>
I20260927 10:50:57.381018 22179 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:57.381191 22179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:57.400663 22179 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:50:57.403569 22226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.405157 22179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:50:57.405292 22179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "a28fdcb17195400997150c29aa3c4f44"
format_stamp: "Formatted at 2026-09-27 10:50:55 on dist-test-slave-p9q6"
server_key: "6f30459348898d9412f487fe233307cc"
server_key_iv: "37f8ba4ae6e71cd8cac7aa2687b1f2f2"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:57.405984 22179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:57.425748 22179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:57.429250 22179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:57.429450 22179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:57.430212 22179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:57.431314 22179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:57.431416 22179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.431505 22179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:57.431562 22179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.443372 22179 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:37999
I20260927 10:50:57.444375 22179 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:50:57.445490 22339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:37999 every 8 connection(s)
I20260927 10:50:57.450003 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22179
I20260927 10:50:57.450482 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:45367
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:40743
--webserver_port=43685
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:57.464625 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.446768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40743 (local address 127.9.39.129:57227)
0927 10:50:57.447267 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:57.447317 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:57.448130 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:57.449466 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:57.449473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:57.449807 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:57.450437 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:57.450457 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.452424 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.452429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:57.453913 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:57.453927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.455312 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.455317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:57.455550 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.456272 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:57.456298 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:57.458137 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:57.460839 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:57.460847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:57.460864 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:57.461238 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:57.461697 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:57.461703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:57.461705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:57.461913 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:57.462798 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:57.462806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:57.463144 (+ 338us) client_negotiation.cc:769] Sending connection context
0927 10:50:57.463357 (+ 213us) client_negotiation.cc:240] Negotiation successful
0927 10:50:57.463670 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":120,"threads_started":1}
I20260927 10:50:57.466341 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.446620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:57227 (local address 127.9.39.190:40743)
0927 10:50:57.448527 (+ 1907us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:57.448551 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:57.448584 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:50:57.448803 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:57.448812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:57.449103 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:57.449401 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:57.450639 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.452256 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.454069 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.454895 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.455076 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.458368 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:57.458376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:57.458393 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:57.458429 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:57.460699 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.461383 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.461391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.461396 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.461544 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.462052 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.462059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.462063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.462568 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:57.462723 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:57.465211 (+ 2488us) server_negotiation.cc:299] Negotiation successful
0927 10:50:57.465477 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1751,"thread_start_us":115,"threads_started":1}
I20260927 10:50:57.468844 22340 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40743
I20260927 10:50:57.469129 22340 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.469678 22340 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
W20260927 10:50:57.471997 22340 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:40743 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 10:50:57.472138 22340 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.472304 22340 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:57.473110 22340 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.473280 22340 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
W20260927 10:50:57.474020 22340 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:50:57.582636 22344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:57.582933 22344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:57.583025 22344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:57.587357 22344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:57.587443 22344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:57.587527 22344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:50:57.592918 22344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:45367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=43685
--tserver_master_addrs=127.9.39.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:57.593981 22344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:57.595062 22344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:57.605013 22349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:57.605010 22350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.605746 22344 server_base.cc:1034] running on GCE node
W20260927 10:50:57.606876 22352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.607268 22344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:57.607966 22344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:57.609164 22344 hybrid_clock.cc:648] HybridClock initialized: now 1790506257609136 us; error 45 us; skew 500 ppm
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:57.612201 22344 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:50:57.613076 22344 webserver.cc:460] Webserver started at http://127.9.39.130:43685/ using document root <none> and password file <none>
I20260927 10:50:57.613621 22344 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:57.613718 22344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:57.634581 22344 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:50:57.637368 22360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.638769 22344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:50:57.638899 22344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "34e5da7f38284565b506a87df496c400"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f5c0e93fd0560272651ba12c7d1891b2"
server_key_iv: "c79ec15a879c9cc020eaca45621ca44f"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:57.639465 22344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:57.671561 22344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:57.675109 22344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:57.675313 22344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:57.675933 22344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:57.676936 22344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:57.677050 22344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.677146 22344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:57.677194 22344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.688426 22344 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:45367
I20260927 10:50:57.688486 22473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:45367 every 8 connection(s)
I20260927 10:50:57.689792 22344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:50:57.698788 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22344
I20260927 10:50:57.699327 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.39.148:39847/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:41085
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:40743
--webserver_port=39229
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:57.711225 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.692915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40743 (local address 127.9.39.130:56647)
0927 10:50:57.693624 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:57.693680 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:57.694779 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:57.695400 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:57.695409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:57.695749 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:57.696430 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:57.696450 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.697577 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.697581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:57.699060 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:57.699072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.700121 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.700130 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:57.700336 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.701073 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:57.701093 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:57.702632 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:57.704756 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:57.704762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:57.704777 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:57.708101 (+ 3324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:57.708448 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:57.708451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:57.708453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:57.708646 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:57.709193 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:57.709200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:57.709518 (+ 318us) client_negotiation.cc:769] Sending connection context
0927 10:50:57.709736 (+ 218us) client_negotiation.cc:240] Negotiation successful
0927 10:50:57.710108 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":126,"threads_started":1}
I20260927 10:50:57.711930 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.694009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:56647 (local address 127.9.39.190:40743)
0927 10:50:57.694150 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:57.694155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:57.694172 (+ 17us) server_negotiation.cc:407] Connection header received
0927 10:50:57.695047 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:57.695051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:57.695108 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:57.695199 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:57.696604 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.697437 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.699224 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.699587 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.699665 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.702813 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:57.702817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:57.702820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:57.702849 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:57.704621 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.708231 (+ 3610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.708235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.708237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.708335 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.708764 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.708768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.708772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.709011 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:57.709131 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:57.711640 (+ 2509us) server_negotiation.cc:299] Negotiation successful
0927 10:50:57.711775 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:50:57.713855 22474 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40743
I20260927 10:50:57.714108 22474 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.714641 22474 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
W20260927 10:50:57.716284 22474 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:40743 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 10:50:57.716403 22474 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.716557 22474 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:57.717371 22474 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.717530 22474 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
W20260927 10:50:57.718250 22474 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:50:57.849684 22477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:50:57.849960 22477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:50:57.850061 22477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:50:57.854411 22477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:50:57.854496 22477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:50:57.854588 22477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:50:57.862106 22477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.39.148:39847/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:41085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=39229
--tserver_master_addrs=127.9.39.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:50:57.863796 22477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:50:57.865160 22477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:50:57.880604 22483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:57.880825 22484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:50:57.881176 22486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:50:57.882889 22477 server_base.cc:1034] running on GCE node
I20260927 10:50:57.883409 22477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:50:57.884188 22477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:50:57.885391 22477 hybrid_clock.cc:648] HybridClock initialized: now 1790506257885361 us; error 45 us; skew 500 ppm
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:57.889099 22477 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:50:57.890266 22477 webserver.cc:460] Webserver started at http://127.9.39.131:39229/ using document root <none> and password file <none>
I20260927 10:50:57.891013 22477 fs_manager.cc:359] Metadata directory not provided
I20260927 10:50:57.891110 22477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:57.909294 22477 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 10:50:57.912544 22494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.913931 22477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:50:57.914059 22477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "2276691b0c294d3cb8596ad06b2d38e3"
format_stamp: "Formatted at 2026-09-27 10:50:56 on dist-test-slave-p9q6"
server_key: "f72a63d45572401bff9c7fecc7b72c7c"
server_key_iv: "1847516e64e4f356f1d87076e4f4a24c"
server_key_version: "kuduclusterkey@0"
I20260927 10:50:57.914629 22477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:50:57.948796 22477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:50:57.952574 22477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:50:57.952836 22477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:50:57.953652 22477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:50:57.954558 22477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:50:57.954625 22477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.954736 22477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:50:57.954789 22477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:50:57.969084 22477 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:41085
I20260927 10:50:57.969353 22607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:41085 every 8 connection(s)
I20260927 10:50:57.970340 22477 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:50:57.973203 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22477
I20260927 10:50:57.982741 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.974185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.9.39.190:40743)
0927 10:50:57.974842 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:57.974847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:57.974861 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:50:57.974908 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:57.974910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:57.974979 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:57.975082 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:57.975895 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.976434 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.977345 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.977586 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.977686 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.979195 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:57.979199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:57.979201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:57.979228 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:57.980896 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.981346 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.981352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.981354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.981468 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.981763 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.981768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.981770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.981937 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:57.982153 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:57.982309 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 10:50:57.982427 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":61,"threads_started":1}
Sep 27 10:50:57 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:50:57.989753 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.974088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:37587 (local address 127.9.39.190:40743)
0927 10:50:57.974228 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:57.974232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:57.974745 (+ 513us) server_negotiation.cc:407] Connection header received
0927 10:50:57.976401 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:57.976405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:57.976450 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:57.976503 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:57.978058 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.978845 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.980218 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.980550 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.980654 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.984090 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:57.984094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:57.984099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:57.984123 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:57.987682 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.988160 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.988163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.988165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.988270 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:57.988626 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:57.988629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:57.988631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:57.988843 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:57.988961 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:57.989489 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 10:50:57.989603 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 10:50:57.990669 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:57.972967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40743 (local address 127.9.39.131:37587)
0927 10:50:57.974559 (+ 1592us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:57.974613 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:57.976144 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:57.976897 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:57.976908 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:57.977248 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:50:57.977871 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:57.977888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.978967 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.978974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:57.980094 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:57.980103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:57.980826 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:57.980831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:57.981056 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:57.982226 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:50:57.982247 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:50:57.983892 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:50:57.987792 (+ 3900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:57.987796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:57.987808 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:57.988062 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:57.988375 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:50:57.988382 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:50:57.988384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:50:57.988520 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:50:57.988961 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:50:57.988967 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:50:57.989252 (+ 285us) client_negotiation.cc:769] Sending connection context
0927 10:50:57.989412 (+ 160us) client_negotiation.cc:240] Negotiation successful
0927 10:50:57.989647 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1369,"thread_start_us":132,"threads_started":1}
I20260927 10:50:57.991933 22608 heartbeater.cc:347] Connected to a master server at 127.9.39.190:40743
I20260927 10:50:57.992201 22608 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.992787 22608 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
W20260927 10:50:57.994376 22608 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:40743 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 10:50:57.994524 22608 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:57.994752 22608 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:57.995707 22141 ts_manager.cc:194] Registered new tserver with Master: 2276691b0c294d3cb8596ad06b2d38e3 (127.9.39.131:41085)
I20260927 10:50:57.996974 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:37587
Sep 27 10:50:58 dist-test-slave-p9q6 krb5kdc[21145](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790506258, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:50:58.474267 22340 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:58.474632 22340 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:58.475749 22141 ts_manager.cc:194] Registered new tserver with Master: a28fdcb17195400997150c29aa3c4f44 (127.9.39.129:37999)
I20260927 10:50:58.476445 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:57227
I20260927 10:50:58.718492 22474 heartbeater.cc:464] Registering TS with master...
I20260927 10:50:58.718832 22474 heartbeater.cc:511] Master 127.9.39.190:40743 requested a full tablet report, sending...
I20260927 10:50:58.719667 22141 ts_manager.cc:194] Registered new tserver with Master: 34e5da7f38284565b506a87df496c400 (127.9.39.130:45367)
I20260927 10:50:58.720454 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:56647
I20260927 10:50:58.731652 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:50:58.743495 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.733253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.9.39.190:40743)
0927 10:50:58.733780 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.733784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.734087 (+ 303us) server_negotiation.cc:407] Connection header received
0927 10:50:58.734127 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.734130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.734173 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.734249 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:50:58.735155 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.735941 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.737130 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.737471 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.737580 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.739310 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:50:58.739314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:50:58.739317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:50:58.739347 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:50:58.741065 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:58.741565 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:58.741568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:58.741570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:58.741660 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:50:58.741984 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:50:58.741988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:50:58.741989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:50:58.742220 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:50:58.742353 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.743076 (+ 723us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.743201 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":94,"threads_started":1}
I20260927 10:50:58.748275 22141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:50:58.752712 22141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 27 10:50:58 dist-test-slave-p9q6 krb5kdc[21145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506258, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for HTTP/127.9.39.148@KRBTEST.COM
I20260927 10:50:58.777590 22635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.771279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45367 (local address 127.0.0.1:39488)
0927 10:50:58.771717 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.771771 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:58.771877 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:58.772524 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:58.772530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:58.772552 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:58.772689 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.772698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.774201 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.774205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.775688 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.775701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.777021 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.777025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.777139 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.777179 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 10:50:58.777316 (+ 137us) client_negotiation.cc:240] Negotiation successful
0927 10:50:58.777438 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":119,"threads_started":1}
I20260927 10:50:58.777633 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.771378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39488 (local address 127.9.39.130:45367)
0927 10:50:58.772106 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.772111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.772129 (+ 18us) server_negotiation.cc:407] Connection header received
0927 10:50:58.772187 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.772191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.772344 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.772494 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:58.772847 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.774062 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.775858 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.776867 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.776993 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.777171 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.777381 (+ 210us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.777491 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":123,"threads_started":1}
I20260927 10:50:58.781253 22390 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4aa9b95a9446feb1995151dda4bed6 (DEFAULT_TABLE table=test-table [id=4455c1d90d684646a5491520c81dc8a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:50:58.782387 22390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4aa9b95a9446feb1995151dda4bed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:58.784444 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.773265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:41085 (local address 127.0.0.1:34514)
0927 10:50:58.774358 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.775196 (+ 838us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:58.775278 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:58.780214 (+ 4936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:58.780218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:58.780235 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:58.780321 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.780327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.781746 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.781749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.782921 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.782932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.784066 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.784069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.784147 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.784167 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:50:58.784229 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 10:50:58.784288 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"thread_start_us":94,"threads_started":1}
I20260927 10:50:58.784873 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.779157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.9.39.131:41085)
0927 10:50:58.779587 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.779594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.779618 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:50:58.779695 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.779701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.779910 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.780069 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:58.780467 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.781602 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.783062 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.783782 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.783879 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.784008 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.784695 (+ 687us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.784772 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":121,"threads_started":1}
I20260927 10:50:58.788776 22541 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4aa9b95a9446feb1995151dda4bed6 (DEFAULT_TABLE table=test-table [id=4455c1d90d684646a5491520c81dc8a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:50:58.789829 22541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4aa9b95a9446feb1995151dda4bed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:58.790642 22641 tablet_bootstrap.cc:492] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: Bootstrap starting.
I20260927 10:50:58.792761 22641 tablet_bootstrap.cc:654] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:58.793464 22641 log.cc:824] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:58.795629 22641 tablet_bootstrap.cc:492] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: No bootstrap required, opened a new log
I20260927 10:50:58.795802 22641 ts_tablet_manager.cc:1397] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260927 10:50:58.800064 22635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.773910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37999 (local address 127.0.0.1:58198)
0927 10:50:58.777738 (+ 3828us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.777752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:58.777816 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:58.795349 (+ 17533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:58.795352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:58.795372 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:58.795479 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.795484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.797351 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.797358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.798495 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.798507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.799655 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.799659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.799752 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.799793 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 10:50:58.799848 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 10:50:58.799932 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3780,"thread_start_us":90,"threads_started":1}
I20260927 10:50:58.800515 22641 raft_consensus.cc:348] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.800804 22641 raft_consensus.cc:374] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:58.800876 22641 raft_consensus.cc:729] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e5da7f38284565b506a87df496c400, State: Initialized, Role: FOLLOWER
I20260927 10:50:58.801335 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.774079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.9.39.129:37999)
0927 10:50:58.794880 (+ 20801us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.794888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.794919 (+ 31us) server_negotiation.cc:407] Connection header received
0927 10:50:58.795000 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.795005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.795199 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.795375 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:58.796133 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.797220 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.798674 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.799515 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.800732 (+ 1217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.801025 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.801073 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.801206 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9757,"thread_start_us":106,"threads_started":1}
I20260927 10:50:58.801313 22641 consensus_queue.cc:260] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.802260 22641 ts_tablet_manager.cc:1428] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:50:58.802879 22474 heartbeater.cc:503] Master 127.9.39.190:40743 was elected leader, sending a full tablet report...
I20260927 10:50:58.804576 22259 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4aa9b95a9446feb1995151dda4bed6 (DEFAULT_TABLE table=test-table [id=4455c1d90d684646a5491520c81dc8a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:50:58.805645 22259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4aa9b95a9446feb1995151dda4bed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:50:58.815879 22643 tablet_bootstrap.cc:492] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Bootstrap starting.
I20260927 10:50:58.818589 22643 tablet_bootstrap.cc:654] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:58.819644 22643 log.cc:824] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:58.823079 22645 tablet_bootstrap.cc:492] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44: Bootstrap starting.
I20260927 10:50:58.825639 22645 tablet_bootstrap.cc:654] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44: Neither blocks nor log segments found. Creating new log.
I20260927 10:50:58.826474 22645 log.cc:824] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44: Log is configured to *not* fsync() on all Append() calls
I20260927 10:50:58.828758 22643 tablet_bootstrap.cc:492] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: No bootstrap required, opened a new log
I20260927 10:50:58.829015 22643 ts_tablet_manager.cc:1397] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260927 10:50:58.834107 22645 tablet_bootstrap.cc:492] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44: No bootstrap required, opened a new log
I20260927 10:50:58.834400 22645 ts_tablet_manager.cc:1397] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260927 10:50:58.834496 22643 raft_consensus.cc:348] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.834837 22643 raft_consensus.cc:374] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:58.834905 22643 raft_consensus.cc:729] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2276691b0c294d3cb8596ad06b2d38e3, State: Initialized, Role: FOLLOWER
I20260927 10:50:58.835412 22643 consensus_queue.cc:260] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.836136 22608 heartbeater.cc:503] Master 127.9.39.190:40743 was elected leader, sending a full tablet report...
I20260927 10:50:58.836552 22643 ts_tablet_manager.cc:1428] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 10:50:58.838999 22645 raft_consensus.cc:348] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.842801 22645 raft_consensus.cc:374] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:50:58.842919 22645 raft_consensus.cc:729] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a28fdcb17195400997150c29aa3c4f44, State: Initialized, Role: FOLLOWER
I20260927 10:50:58.843410 22645 consensus_queue.cc:260] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.844295 22340 heartbeater.cc:503] Master 127.9.39.190:40743 was elected leader, sending a full tablet report...
I20260927 10:50:58.844609 22645 ts_tablet_manager.cc:1428] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260927 10:50:58.874675 22648 raft_consensus.cc:482] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:50:58.874976 22648 raft_consensus.cc:504] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.876219 22648 leader_election.cc:288] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e5da7f38284565b506a87df496c400 (127.9.39.130:45367), a28fdcb17195400997150c29aa3c4f44 (127.9.39.129:37999)
I20260927 10:50:58.881609 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.876867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:57789 (local address 127.9.39.130:45367)
0927 10:50:58.876989 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.876993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.877008 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:50:58.877158 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.877164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.877222 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.877275 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:58.877733 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.878543 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.880411 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.880987 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.881072 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.881127 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.881418 (+ 291us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.881494 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:50:58.881609 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.876670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:39767 (local address 127.9.39.129:37999)
0927 10:50:58.876800 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.876805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.877216 (+ 411us) server_negotiation.cc:407] Connection header received
0927 10:50:58.877395 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.877402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.877453 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.877500 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:50:58.877885 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.878596 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.880281 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.880889 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.880979 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.881042 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.881405 (+ 363us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.881482 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:50:58.882047 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.876519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37999 (local address 127.9.39.131:39767)
0927 10:50:58.877141 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.877154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:58.877278 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:58.877620 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:58.877622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:58.877638 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:58.877758 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.877763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.878739 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.878745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.880138 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.880150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.881139 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.881145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.881249 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.881281 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 10:50:58.881809 (+ 528us) client_negotiation.cc:240] Negotiation successful
0927 10:50:58.881900 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":65,"thread_start_us":80,"threads_started":1}
I20260927 10:50:58.882234 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.876417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45367 (local address 127.9.39.131:57789)
0927 10:50:58.876889 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.876905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:50:58.877018 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:50:58.877443 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:50:58.877447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:50:58.877462 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:50:58.877609 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.877615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.878710 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.878715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.880271 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:50:58.880279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.881199 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.881202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:50:58.881281 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.881297 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 10:50:58.882009 (+ 712us) client_negotiation.cc:240] Negotiation successful
0927 10:50:58.882080 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":99,"thread_start_us":128,"threads_started":1}
I20260927 10:50:58.882862 22294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4aa9b95a9446feb1995151dda4bed6" candidate_uuid: "2276691b0c294d3cb8596ad06b2d38e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a28fdcb17195400997150c29aa3c4f44" is_pre_election: true
I20260927 10:50:58.883090 22428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4aa9b95a9446feb1995151dda4bed6" candidate_uuid: "2276691b0c294d3cb8596ad06b2d38e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e5da7f38284565b506a87df496c400" is_pre_election: true
I20260927 10:50:58.883221 22294 raft_consensus.cc:2454] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2276691b0c294d3cb8596ad06b2d38e3 in term 0.
I20260927 10:50:58.883360 22428 raft_consensus.cc:2454] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2276691b0c294d3cb8596ad06b2d38e3 in term 0.
I20260927 10:50:58.883720 22495 leader_election.cc:302] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2276691b0c294d3cb8596ad06b2d38e3, a28fdcb17195400997150c29aa3c4f44; no voters:
I20260927 10:50:58.883975 22648 raft_consensus.cc:2781] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:50:58.884039 22648 raft_consensus.cc:482] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:50:58.884078 22648 raft_consensus.cc:3037] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:58.885263 22648 raft_consensus.cc:504] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.885658 22648 leader_election.cc:288] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [CANDIDATE]: Term 1 election: Requested vote from peers 34e5da7f38284565b506a87df496c400 (127.9.39.130:45367), a28fdcb17195400997150c29aa3c4f44 (127.9.39.129:37999)
I20260927 10:50:58.885957 22428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4aa9b95a9446feb1995151dda4bed6" candidate_uuid: "2276691b0c294d3cb8596ad06b2d38e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e5da7f38284565b506a87df496c400"
I20260927 10:50:58.886026 22294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4aa9b95a9446feb1995151dda4bed6" candidate_uuid: "2276691b0c294d3cb8596ad06b2d38e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a28fdcb17195400997150c29aa3c4f44"
I20260927 10:50:58.886078 22428 raft_consensus.cc:3037] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:58.886170 22294 raft_consensus.cc:3037] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:50:58.887174 22294 raft_consensus.cc:2454] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2276691b0c294d3cb8596ad06b2d38e3 in term 1.
I20260927 10:50:58.887394 22428 raft_consensus.cc:2454] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2276691b0c294d3cb8596ad06b2d38e3 in term 1.
I20260927 10:50:58.887420 22495 leader_election.cc:302] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2276691b0c294d3cb8596ad06b2d38e3, a28fdcb17195400997150c29aa3c4f44; no voters:
I20260927 10:50:58.887661 22648 raft_consensus.cc:2781] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:50:58.887921 22648 raft_consensus.cc:686] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 1 LEADER]: Becoming Leader. State: Replica: 2276691b0c294d3cb8596ad06b2d38e3, State: Running, Role: LEADER
I20260927 10:50:58.888283 22648 consensus_queue.cc:237] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } }
I20260927 10:50:58.892069 22141 catalog_manager.cc:5495] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2276691b0c294d3cb8596ad06b2d38e3 (127.9.39.131). New cstate: current_term: 1 leader_uuid: "2276691b0c294d3cb8596ad06b2d38e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2276691b0c294d3cb8596ad06b2d38e3" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 } health_report { overall_health: UNKNOWN } } }
I20260927 10:50:58.913427 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.908693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.9.39.131:41085)
0927 10:50:58.908821 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:50:58.908825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:50:58.908839 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:50:58.908918 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:50:58.908921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:50:58.908985 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:50:58.909108 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:50:58.909406 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.910196 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.911545 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:50:58.911881 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:50:58.911982 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:50:58.912376 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:50:58.912934 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:50:58.913048 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 10:50:58.913219 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 10:50:58.913288 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 10:50:58.920342 22428 raft_consensus.cc:1261] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 1 FOLLOWER]: Refusing update from remote peer 2276691b0c294d3cb8596ad06b2d38e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:50:58.920369 22294 raft_consensus.cc:1261] T 8a4aa9b95a9446feb1995151dda4bed6 P a28fdcb17195400997150c29aa3c4f44 [term 1 FOLLOWER]: Refusing update from remote peer 2276691b0c294d3cb8596ad06b2d38e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:50:58.921141 22652 consensus_queue.cc:1035] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e5da7f38284565b506a87df496c400" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 45367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:50:58.921286 22648 consensus_queue.cc:1035] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a28fdcb17195400997150c29aa3c4f44" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:50:58.933568 22656 mvcc.cc:205] Tried to move back new op lower bound from 7333913636529176576 to 7333913636409470976. Current Snapshot: MvccSnapshot[applied={T|T < 7333913636529176576}]
I20260927 10:50:58.941066 22141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:50:58.941226 22141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:50:58.943871 22141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0ac31535a484bff8267b6a55c91615d: Sending DeleteTablet for 3 replicas of tablet 8a4aa9b95a9446feb1995151dda4bed6
I20260927 10:50:58.944725 22390 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4aa9b95a9446feb1995151dda4bed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:50:58 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:39488
I20260927 10:50:58.945056 22541 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4aa9b95a9446feb1995151dda4bed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:50:58 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34514
I20260927 10:50:58.945369 22259 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4aa9b95a9446feb1995151dda4bed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:50:58 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:58198
I20260927 10:50:58.947201 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22179
I20260927 10:50:58.954964 22665 tablet_replica.cc:330] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: stopping tablet replica
I20260927 10:50:58.955266 22664 tablet_replica.cc:330] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: stopping tablet replica
I20260927 10:50:58.955739 22664 raft_consensus.cc:2229] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:50:58.956082 22665 raft_consensus.cc:2229] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:50:58.956569 22665 raft_consensus.cc:2258] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:50:58.956969 22664 raft_consensus.cc:2258] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:50:58.959115 22664 ts_tablet_manager.cc:1905] T 8a4aa9b95a9446feb1995151dda4bed6 P 34e5da7f38284565b506a87df496c400: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 10:50:58.959774 22125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:37999 (error 108)
I20260927 10:50:58.962539 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22344
I20260927 10:50:58.965348 22665 ts_tablet_manager.cc:1905] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:50:58.968299 22665 ts_tablet_manager.cc:1918] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:50:58.968477 22665 log.cc:1196] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/8a4aa9b95a9446feb1995151dda4bed6
I20260927 10:50:58.968923 22665 ts_tablet_manager.cc:1939] T 8a4aa9b95a9446feb1995151dda4bed6 P 2276691b0c294d3cb8596ad06b2d38e3: Deleting consensus metadata
I20260927 10:50:58.970139 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22477
I20260927 10:50:58.970793 22127 catalog_manager.cc:4841] TS 2276691b0c294d3cb8596ad06b2d38e3 (127.9.39.131:41085): tablet 8a4aa9b95a9446feb1995151dda4bed6 (table test-table [id=4455c1d90d684646a5491520c81dc8a7]) successfully deleted
I20260927 10:50:58.971575 22635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.971169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:37999 (local address 127.0.0.1:58208)
0927 10:50:58.971387 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.971468 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260927 10:50:58.971742 22125 catalog_manager.cc:4584] TS a28fdcb17195400997150c29aa3c4f44 (127.9.39.129:37999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a4aa9b95a9446feb1995151dda4bed6: Network error: Client connection negotiation failed: client connection to 127.9.39.129:37999: connect: Connection refused (error 111)
I20260927 10:50:58.972492 22635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:50:58.972195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:45367 (local address 127.0.0.1:39500)
0927 10:50:58.972364 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 10:50:58.972416 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.130:45367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 10:50:58.977792 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22108
I20260927 10:50:58.984689 9374 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 10:50:59.319850 9374 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 10:50:59.319952 9374 mini_ranger.cc:66] Stopping Ranger...
I20260927 10:50:59.664750 9374 mini_ranger.cc:68] Stopped Ranger
2026-09-27 10:50:59.664 UTC [21175] LOG: received smart shutdown request
2026-09-27 10:50:59.668 UTC [21175] LOG: background worker "logical replication launcher" (PID 21184) exited with exit code 1
2026-09-27 10:50:59.669 UTC [21179] LOG: shutting down
2026-09-27 10:50:59.831 UTC [21175] LOG: database system is shut down
2026-09-27T10:50:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47657 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:51:00 dist-test-slave-p9q6 krb5kdc[22676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:51:00 dist-test-slave-p9q6 krb5kdc[22676](info): set up 2 sockets
Sep 27 10:51:00 dist-test-slave-p9q6 krb5kdc[22676](info): commencing operation
krb5kdc: starting...
W20260927 10:51:00.986886 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:51:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:51:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:51:01.106041 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46581
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46581,127.9.39.189:44637,127.9.39.188:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:01.214905 22689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:01.215164 22689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:01.215204 22689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:01.219237 22689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:01.219288 22689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:01.219306 22689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:01.219323 22689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:01.219383 22689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:01.227087 22689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46581,127.9.39.189:44637,127.9.39.188:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:01.228811 22689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:01.230455 22689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:01.242103 22695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:01.242130 22694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:01.242103 22697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:01.242576 22689 server_base.cc:1034] running on GCE node
I20260927 10:51:01.243306 22689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:01.244539 22689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:01.245746 22689 hybrid_clock.cc:648] HybridClock initialized: now 1790506261245711 us; error 52 us; skew 500 ppm
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:01.248848 22689 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:01.249866 22689 webserver.cc:460] Webserver started at http://127.9.39.190:46625/ using document root <none> and password file <none>
I20260927 10:51:01.250646 22689 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:01.250792 22689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:01.251037 22689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:01.253043 22689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "8eb41121018641b899d6d4fe61490bf3"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "2e679c582c1ca9243e2ae03a29832684"
server_key_iv: "935cb975f98e33571286593a0d6ec038"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.253720 22689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "8eb41121018641b899d6d4fe61490bf3"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "2e679c582c1ca9243e2ae03a29832684"
server_key_iv: "935cb975f98e33571286593a0d6ec038"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.258592 22689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260927 10:51:01.261528 22704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:01.263025 22689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:51:01.263199 22689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "8eb41121018641b899d6d4fe61490bf3"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "2e679c582c1ca9243e2ae03a29832684"
server_key_iv: "935cb975f98e33571286593a0d6ec038"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.263411 22689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:01.276515 22689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:01.279253 22689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:01.279397 22689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:01.286484 22689 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:46581
I20260927 10:51:01.286566 22756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:46581 every 8 connection(s)
I20260927 10:51:01.287438 22689 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:01.290189 22757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:01.292016 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22689
I20260927 10:51:01.292155 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:51:01.292429 9374 external_mini_cluster.cc:1428] Setting key 044db6720636830e1400ca1003a90cae
I20260927 10:51:01.295037 22757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.39.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.189@KRBTEST.COM" created.
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:01.310869 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.297645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.9.39.190:46581)
0927 10:51:01.298038 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.298057 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.298318 (+ 261us) server_negotiation.cc:407] Connection header received
0927 10:51:01.299361 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.299365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.299545 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.299719 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.300527 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.301472 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.302539 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.302969 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.303160 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.305743 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.305754 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.305771 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.305803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.307871 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.308460 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.308464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.308466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.308570 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.308933 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.308935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.308936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.309285 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.309384 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.309863 (+ 479us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.310056 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":128,"threads_started":1}
I20260927 10:51:01.310871 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.297646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46581 (local address 127.0.0.1:53316)
0927 10:51:01.298184 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.298218 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.299068 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.299715 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.299721 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.300147 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.300369 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.300381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.301603 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.301606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.302381 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.302390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.303140 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.303144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.303250 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.303841 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.303857 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.305559 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.307987 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.307998 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.308010 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.308347 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.308667 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.308673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.308677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.308838 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.309404 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.309419 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.309606 (+ 187us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.309810 (+ 204us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.310183 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":51,"spinlock_wait_cycles":896,"thread_start_us":143,"threads_started":1}
I20260927 10:51:01.313031 22757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:51:01.313766 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.313482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53130)
0927 10:51:01.313630 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.313711 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260927 10:51:01.313953 22707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
I20260927 10:51:01.314800 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.314529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53146)
0927 10:51:01.314640 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.314737 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 10:51:01.314944 22757 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:44637: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab.
Entry for principal kudu/127.9.39.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab.
I20260927 10:51:01.319382 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:44637
--webserver_interface=127.9.39.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab
--principal=kudu/127.9.39.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46581,127.9.39.189:44637,127.9.39.188:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:01.364193 22757 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } attempt: 1
I20260927 10:51:01.365097 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.364770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53152)
0927 10:51:01.364919 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.365010 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 10:51:01.365790 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.365587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53160)
0927 10:51:01.365688 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.365737 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 10:51:01.365916 22757 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:44637: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
W20260927 10:51:01.425276 22764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:01.425544 22764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:01.425630 22764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:01.429658 22764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:01.429735 22764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:01.429771 22764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:01.429828 22764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:01.429865 22764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:01.434154 22757 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } attempt: 2
I20260927 10:51:01.434984 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.434677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53162)
0927 10:51:01.434850 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.434915 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260927 10:51:01.435175 22764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46581,127.9.39.189:44637,127.9.39.188:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab
--principal=kudu/127.9.39.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:44637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.39.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:01.435734 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.435525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53174)
0927 10:51:01.435633 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.435682 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260927 10:51:01.435858 22757 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:44637: Network error: Client connection negotiation failed: client connection to 127.9.39.189:44637: connect: Connection refused (error 111)
I20260927 10:51:01.436246 22764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:01.437189 22764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:01.445441 22772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:01.445504 22769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:01.445530 22764 server_base.cc:1034] running on GCE node
W20260927 10:51:01.445446 22770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:01.446205 22764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:01.447206 22764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:01.448359 22764 hybrid_clock.cc:648] HybridClock initialized: now 1790506261448346 us; error 23 us; skew 500 ppm
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:01.454209 22764 init.cc:377] Logged in from keytab as kudu/127.9.39.189@KRBTEST.COM (short username kudu)
I20260927 10:51:01.455149 22764 webserver.cc:460] Webserver started at http://127.9.39.189:36307/ using document root <none> and password file <none>
I20260927 10:51:01.455691 22764 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:01.455775 22764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:01.455991 22764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:01.457919 22764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data/instance:
uuid: "8c0638ffc60542228eb762af1e347b1c"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "e6163ce66eb9764ff2ae2cee975e66ca"
server_key_iv: "9cf46132c223db56aff8c66de842588e"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.458472 22764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal/instance:
uuid: "8c0638ffc60542228eb762af1e347b1c"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "e6163ce66eb9764ff2ae2cee975e66ca"
server_key_iv: "9cf46132c223db56aff8c66de842588e"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.461992 22764 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:51:01.464285 22779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:01.465404 22764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:51:01.465546 22764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal
uuid: "8c0638ffc60542228eb762af1e347b1c"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "e6163ce66eb9764ff2ae2cee975e66ca"
server_key_iv: "9cf46132c223db56aff8c66de842588e"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.465651 22764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:01.486052 22764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:01.488922 22764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:01.489125 22764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:01.496260 22764 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.189:44637
I20260927 10:51:01.496289 22831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.189:44637 every 8 connection(s)
I20260927 10:51:01.497175 22764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
I20260927 10:51:01.499785 22832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:01.503362 22832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:51:01.505643 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22764
I20260927 10:51:01.505798 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-1/wal/instance
I20260927 10:51:01.506099 9374 external_mini_cluster.cc:1428] Setting key cc3c16cc44935c65d88406c4bd744ce0
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.188@KRBTEST.COM; defaulting to no policy
I20260927 10:51:01.517518 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.505192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.9.39.190:46581)
0927 10:51:01.505327 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.505331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.506230 (+ 899us) server_negotiation.cc:407] Connection header received
0927 10:51:01.507621 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.507624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.507694 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.507800 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.508919 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.509798 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.511198 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.511429 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.511515 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.514311 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.514315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.514318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.514346 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.515748 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.516214 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.516216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.516217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.516286 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.516563 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.516565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.516566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.516731 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.516805 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.517287 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.517395 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.188@KRBTEST.COM" created.
I20260927 10:51:01.518501 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.505556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46581 (local address 127.0.0.1:53318)
0927 10:51:01.506063 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.506125 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.507404 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.507988 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.507994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.508309 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.508763 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.508775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.509924 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.509926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.511075 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.511089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.511591 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.511598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.511841 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.512485 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.512511 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.514139 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.515863 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.515867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.515879 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.516131 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.516361 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.516363 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.516365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.516487 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.516815 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.516819 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.517059 (+ 240us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.517221 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.517479 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":136,"threads_started":1}
I20260927 10:51:01.519956 22832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:51:01.527043 22757 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } attempt: 3
I20260927 10:51:01.530047 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.520457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53186)
0927 10:51:01.520584 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.520599 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.520671 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.521443 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.521445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.521576 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.521658 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.521663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.523163 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.523165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.523679 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.523687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.524530 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.524532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.524638 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.525117 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.525129 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.526245 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.528533 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.528535 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.528536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.528708 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.529004 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.529007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.529009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.529110 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.529580 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.529583 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.529760 (+ 177us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.529840 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.529929 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":69}
I20260927 10:51:01.530527 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.520633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.9.39.189:44637)
0927 10:51:01.521015 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.521035 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.521059 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:51:01.521108 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.521116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.521309 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.521448 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.521764 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.523046 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.523789 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.524405 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.524566 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.526377 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.526388 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.526402 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.526432 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.528431 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.528813 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.528820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.528822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.528915 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.529219 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.529225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.529230 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.529473 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.529581 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.529957 (+ 376us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.530096 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":266,"thread_start_us":164,"threads_started":1}
I20260927 10:51:01.532496 22832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:51:01.533306 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.532966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55128)
0927 10:51:01.533097 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.533198 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260927 10:51:01.533763 22783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
I20260927 10:51:01.534906 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.534577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55144)
0927 10:51:01.534739 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.534823 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 10:51:01.535087 22832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:45405: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
I20260927 10:51:01.536567 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.527580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.9.39.189:44637)
0927 10:51:01.527877 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.527884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.527895 (+ 11us) server_negotiation.cc:407] Connection header received
0927 10:51:01.527921 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.527923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.527953 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.528067 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.528959 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.529630 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.530750 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.531018 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.531104 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.533158 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.533164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.533169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.533192 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.534810 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.535277 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.535281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.535283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.535377 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.535695 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.535698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.535699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.535926 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.536042 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.536307 (+ 265us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.536411 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":101,"threads_started":1}
I20260927 10:51:01.536589 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.527491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53190)
0927 10:51:01.527606 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.527623 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.527704 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.528109 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.528112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.528290 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.528845 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.528857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.529733 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.529736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.530578 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.530590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.531134 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.531138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.531250 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.531712 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.531732 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.533015 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.534911 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.534913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.534915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.535194 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.535464 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.535467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.535469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.535580 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.536037 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.536041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.536141 (+ 100us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.536247 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.536434 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab.
Entry for principal kudu/127.9.39.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab.
I20260927 10:51:01.537657 22757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:51:01.537930 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:45405
--webserver_interface=127.9.39.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab
--principal=kudu/127.9.39.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46581,127.9.39.189:44637,127.9.39.188:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:01.538426 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.538110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55148)
0927 10:51:01.538286 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.538342 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260927 10:51:01.539098 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.538873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55150)
0927 10:51:01.539009 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.539051 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260927 10:51:01.539265 22757 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:45405: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
I20260927 10:51:01.570430 22757 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } attempt: 1
I20260927 10:51:01.571306 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.570966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55156)
0927 10:51:01.571142 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.571213 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260927 10:51:01.572062 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.571816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55164)
0927 10:51:01.571953 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.572002 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 10:51:01.572227 22757 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:45405: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
I20260927 10:51:01.584329 22832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } attempt: 1
I20260927 10:51:01.585320 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.584901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55170)
0927 10:51:01.585099 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.585188 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260927 10:51:01.586051 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.585834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55172)
0927 10:51:01.585939 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.585989 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 10:51:01.586182 22832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:45405: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
I20260927 10:51:01.647394 22757 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } attempt: 2
W20260927 10:51:01.647293 22840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:01.647523 22840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:01.647563 22840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 10:51:01.648260 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.647952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55188)
0927 10:51:01.648124 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.648186 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260927 10:51:01.648940 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.648755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55192)
0927 10:51:01.648845 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.648886 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 10:51:01.649173 22757 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:45405: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
W20260927 10:51:01.652184 22840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:01.652236 22840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:01.652256 22840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:01.652308 22840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:01.652356 22840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:01.654338 22832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } attempt: 2
I20260927 10:51:01.655031 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.654779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55206)
0927 10:51:01.654904 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.654959 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 10:51:01.655727 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.655524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55208)
0927 10:51:01.655623 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.655669 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 10:51:01.655848 22832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:45405: Network error: Client connection negotiation failed: client connection to 127.9.39.188:45405: connect: Connection refused (error 111)
I20260927 10:51:01.658252 22840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46581,127.9.39.189:44637,127.9.39.188:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab
--principal=kudu/127.9.39.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.39.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:01.659334 22840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:01.660359 22840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:01.668602 22845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:01.668651 22846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:01.668766 22840 server_base.cc:1034] running on GCE node
W20260927 10:51:01.668951 22848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:01.669581 22840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:01.670567 22840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:01.671721 22840 hybrid_clock.cc:648] HybridClock initialized: now 1790506261671704 us; error 27 us; skew 500 ppm
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:01.674427 22840 init.cc:377] Logged in from keytab as kudu/127.9.39.188@KRBTEST.COM (short username kudu)
I20260927 10:51:01.675294 22840 webserver.cc:460] Webserver started at http://127.9.39.188:34609/ using document root <none> and password file <none>
I20260927 10:51:01.675794 22840 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:01.675849 22840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:01.676007 22840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:01.677902 22840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data/instance:
uuid: "4011d3c6981048f6a022680d1ff22ceb"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "0df85f63b5bfd74cb9f9cdd23654012c"
server_key_iv: "45053c47f1bfd678052c9ba86d268f48"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.678399 22840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal/instance:
uuid: "4011d3c6981048f6a022680d1ff22ceb"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "0df85f63b5bfd74cb9f9cdd23654012c"
server_key_iv: "45053c47f1bfd678052c9ba86d268f48"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.681883 22840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:51:01.684162 22855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:01.685303 22840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:51:01.685425 22840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal
uuid: "4011d3c6981048f6a022680d1ff22ceb"
format_stamp: "Formatted at 2026-09-27 10:51:01 on dist-test-slave-p9q6"
server_key: "0df85f63b5bfd74cb9f9cdd23654012c"
server_key_iv: "45053c47f1bfd678052c9ba86d268f48"
server_key_version: "encryptionkey@0"
I20260927 10:51:01.685535 22840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:01.719635 22840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:01.723047 22840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:01.723213 22840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:01.730356 22840 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.188:45405
I20260927 10:51:01.730376 22907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.188:45405 every 8 connection(s)
I20260927 10:51:01.731302 22840 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
I20260927 10:51:01.733811 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:01.734485 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22840
I20260927 10:51:01.734624 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790506104903010-9374-0/minicluster-data/master-2/wal/instance
I20260927 10:51:01.734934 9374 external_mini_cluster.cc:1428] Setting key 27d275499f95fd6693d3e7f81c7e2b06
I20260927 10:51:01.737905 22908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:01.747018 22832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } attempt: 3
I20260927 10:51:01.747967 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.739598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.9.39.190:46581)
0927 10:51:01.739743 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.739747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.740017 (+ 270us) server_negotiation.cc:407] Connection header received
0927 10:51:01.740146 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.740149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.740207 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.740306 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.740920 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.741379 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.742212 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.742415 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.742544 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.744871 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.744874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.744877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.744906 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.746160 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.746585 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.746589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.746591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.746681 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.747011 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.747014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.747016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.747227 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.747323 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.747722 (+ 399us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.747844 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 10:51:01.748377 22757 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } attempt: 3
I20260927 10:51:01.749493 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.739496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46581 (local address 127.0.0.1:53330)
0927 10:51:01.739892 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.739938 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.740049 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.740350 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.740356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.740685 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.740803 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.740808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.741490 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.741497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.742099 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.742107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.742586 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.742590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.742736 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.743267 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.743288 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.744738 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.746276 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.746285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.746293 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.746495 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.746791 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.746795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.746797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.746903 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.747324 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.747331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.747474 (+ 143us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.748178 (+ 704us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.748490 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":153,"threads_started":1}
I20260927 10:51:01.750066 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.736298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.9.39.188:45405)
0927 10:51:01.736712 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.736726 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.736760 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:51:01.737393 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.737409 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.737738 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.738069 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.738902 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.740156 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.741034 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.741494 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.741664 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.744297 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.744303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.744315 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.744358 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.746183 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.746598 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.746603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.746604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.746776 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.747050 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.747056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.747059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.747473 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.748214 (+ 741us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.748318 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.749872 (+ 1554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1402,"server-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260927 10:51:01.751852 22908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
I20260927 10:51:01.757606 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.747410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55228)
0927 10:51:01.747566 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.747582 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.747651 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.748134 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.748137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.748310 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.748408 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.748414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.749562 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.749569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.750345 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.750356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.750961 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.750965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.751050 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.751541 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.751554 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.753045 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.755220 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.755223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.755225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.755489 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.755829 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.755832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.755834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.755931 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.757083 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.757088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.757209 (+ 121us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.757297 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.757417 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"spinlock_wait_cycles":2816}
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:01.758399 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.747651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.9.39.188:45405)
0927 10:51:01.747903 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.747911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.747924 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:51:01.747955 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.747957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.747996 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.748577 (+ 581us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.748655 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.749362 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.750462 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.750831 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.751238 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.753156 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.753160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.753163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.753184 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.755096 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.755644 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.755647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.755649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.755729 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.756033 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.756036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.756037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.756958 (+ 921us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.757990 (+ 1032us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.758059 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.758157 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"spinlock_wait_cycles":1792,"thread_start_us":97,"threads_started":1}
I20260927 10:51:01.759838 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.748738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55230)
0927 10:51:01.748836 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.748849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.748912 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.749576 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.749582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.749757 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.749838 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.749850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.752055 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.752059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.752824 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.752834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.753963 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.753970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.754058 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.754427 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.754439 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.755814 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.757885 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.757887 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.757890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.758124 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.758445 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.758449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.758453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.758556 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.759476 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.759479 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.759576 (+ 97us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.759644 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.759732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":1}
I20260927 10:51:01.761816 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.752194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:44637 (local address 127.0.0.1:53206)
0927 10:51:01.752521 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.752535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.752599 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.753012 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.753014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.753173 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.753273 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.753280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.754384 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.754387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.755239 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.755247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.755959 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.755963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.756081 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.756732 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.756742 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.758538 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.760431 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.760436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.760438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.760620 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.760896 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.760899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.760901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.760985 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.761372 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.761375 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.761461 (+ 86us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.761565 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.761676 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"spinlock_wait_cycles":2432}
I20260927 10:51:01.761847 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.752740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.9.39.189:44637)
0927 10:51:01.752838 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.752841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.752850 (+ 9us) server_negotiation.cc:407] Connection header received
0927 10:51:01.752877 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.752879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.752910 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.752992 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.753419 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.754252 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.755373 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.755738 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.755817 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.758706 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.758710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.758713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.758772 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.760326 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.760714 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.760718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.760720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.760799 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.761077 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.761080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.761082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.761267 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.761380 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.761613 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.761714 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39,"spinlock_wait_cycles":1280}
I20260927 10:51:01.762077 22832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: Bootstrap starting.
I20260927 10:51:01.762374 22908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:51:01.763715 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.748926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.9.39.188:45405)
0927 10:51:01.749228 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.749254 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.749287 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:51:01.749317 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.749319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.749379 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.749930 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.750017 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.751904 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.753432 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.753825 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.754254 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.755940 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.755943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.755946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.755966 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.757767 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.758231 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.758233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.758235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.758319 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.758671 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.758674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.758675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.758892 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.763301 (+ 4409us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.763376 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.763535 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":102,"threads_started":1}
I20260927 10:51:01.765103 22832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:01.766111 22832 log.cc:824] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:01.766286 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.752398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.9.39.190:46581)
0927 10:51:01.752516 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.752520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.753165 (+ 645us) server_negotiation.cc:407] Connection header received
0927 10:51:01.753202 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.753204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.753234 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.753324 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.753991 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.754643 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.755855 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.756199 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.756290 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.759038 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.759042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.759046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.759070 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.764142 (+ 5072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.764612 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.764615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.764617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.764698 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.764996 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.764999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.765001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.765234 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.765442 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.766036 (+ 594us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.766142 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:51:01.766607 22757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: Bootstrap starting.
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
I20260927 10:51:01.769297 22757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:01.770159 22757 log.cc:824] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:01.771780 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.762812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:45405 (local address 127.0.0.1:55242)
0927 10:51:01.762954 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:01.762969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:01.763048 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:01.763540 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:01.763543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:01.763690 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:01.763979 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.763987 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.764892 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.764894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.765653 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:01.765664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.766262 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.766265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:01.766328 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.766790 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:01.766803 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:01.768131 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:01.770016 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.770019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.770021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.770290 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.770570 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:01.770572 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:01.770574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:01.770650 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:01.771054 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:01.771057 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:01.771133 (+ 76us) client_negotiation.cc:769] Sending connection context
0927 10:51:01.771465 (+ 332us) client_negotiation.cc:240] Negotiation successful
0927 10:51:01.771623 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260927 10:51:01.771807 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.762846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.9.39.188:45405)
0927 10:51:01.762937 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.762941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.763008 (+ 67us) server_negotiation.cc:407] Connection header received
0927 10:51:01.763131 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.763134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.763170 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.763245 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.764091 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.764787 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.765769 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.766074 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.766165 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.768256 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.768259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.768261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.768280 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.769912 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.770370 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.770375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.770377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.770453 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.770776 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.770778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.770780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.770944 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.771022 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.771533 (+ 511us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.771678 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":40}
I20260927 10:51:01.772351 22757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: No bootstrap required, opened a new log
Sep 27 10:51:01 dist-test-slave-p9q6 krb5kdc[22676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:51:01.775264 22908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb: Bootstrap starting.
I20260927 10:51:01.775815 22757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.776044 22757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:01.776103 22757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb41121018641b899d6d4fe61490bf3, State: Initialized, Role: FOLLOWER
I20260927 10:51:01.776588 22757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.777096 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.777289 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:51:01.777359 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.767506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.9.39.189:44637)
0927 10:51:01.767628 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.767632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.767646 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:51:01.767677 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.767680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.767716 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.767793 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.768710 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.769533 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.770537 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.770897 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.771010 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.773708 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.773711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.773714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.773737 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.775397 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.775852 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.775855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.775857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.775943 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.776241 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.776244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.776246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.776468 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.776653 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.777136 (+ 483us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.777234 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:51:01.777832 22757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:01.778237 22908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:01.779209 22908 log.cc:824] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:01.780524 22832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: No bootstrap required, opened a new log
I20260927 10:51:01.781008 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:51:01.781131 22908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb: No bootstrap required, opened a new log
I20260927 10:51:01.784729 22832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.785074 22832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:01.785184 22832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0638ffc60542228eb762af1e347b1c, State: Initialized, Role: FOLLOWER
I20260927 10:51:01.785498 22908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.785714 22908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:01.785782 22908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4011d3c6981048f6a022680d1ff22ceb, State: Initialized, Role: FOLLOWER
I20260927 10:51:01.785683 22832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.787194 22832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:01.789948 22908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.791934 22908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:01.789484 22689 master_runner.cc:186] Error getting master registration for 127.9.39.189:44637: OK, instance_id { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" instance_seqno: 1790506261494197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 10:51:01.792984 22689 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 10:51:01.783316 22929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:51:01.786154 22710 tablet.cc:2369] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:51:01.796686 22929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:51:01.797014 22936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.797178 22936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:51:01.797835 22944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.797967 22944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: This master's current role is: FOLLOWER
W20260927 10:51:01.801407 22960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:51:01.801468 22960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:51:01.797724 22840 master_runner.cc:186] Error getting master registration for 127.9.39.189:44637: OK, instance_id { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" instance_seqno: 1790506261494197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 10:51:01.801613 22840 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:51:01.802961 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.787410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.9.39.190:46581)
0927 10:51:01.787508 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.787515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.787827 (+ 312us) server_negotiation.cc:407] Connection header received
0927 10:51:01.787864 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.787867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.787905 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.787977 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.793978 (+ 6001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.794669 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.797156 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.797465 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.797561 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.799182 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.799186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.799188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.799213 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.800877 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.801342 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.801347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.801349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.801427 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.801739 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.801744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.801748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.801930 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.802088 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.802632 (+ 544us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.802784 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 10:51:01.804148 22961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:51:01.804226 22961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:51:01.804373 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.786418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.9.39.189:44637)
0927 10:51:01.786530 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.786534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.787999 (+ 1465us) server_negotiation.cc:407] Connection header received
0927 10:51:01.788159 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.788162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.788204 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.788269 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.789244 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.790015 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.791337 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.791684 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.791758 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.793561 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.793567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.793570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.793597 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.795267 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.795778 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.795781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.795783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.795863 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.796168 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.796171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.796173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.796409 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.796518 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.804041 (+ 7523us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.804158 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 10:51:01.806818 22764 master_runner.cc:263] No leader master found from master 8c0638ffc60542228eb762af1e347b1c
I20260927 10:51:01.806877 22764 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:51:01.808632 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:01.785983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.9.39.188:45405)
0927 10:51:01.786288 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:01.786293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:01.787562 (+ 1269us) server_negotiation.cc:407] Connection header received
0927 10:51:01.787629 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:01.787632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:01.787675 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:01.787753 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:01.795732 (+ 7979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.796407 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.802924 (+ 6517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:01.803270 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:01.803342 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:01.805212 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:01.805216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:01.805218 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:01.805243 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:01.806978 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.807419 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.807422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.807424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.807513 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:01.807805 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:01.807808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:01.807810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:01.807986 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:01.808148 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:01.808348 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 10:51:01.808458 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260927 10:51:01.863603 22944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:01.863795 22944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.864807 22944 leader_election.cc:288] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb41121018641b899d6d4fe61490bf3 (127.9.39.190:46581), 8c0638ffc60542228eb762af1e347b1c (127.9.39.189:44637)
I20260927 10:51:01.865561 22806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4011d3c6981048f6a022680d1ff22ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0638ffc60542228eb762af1e347b1c" is_pre_election: true
I20260927 10:51:01.865854 22806 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4011d3c6981048f6a022680d1ff22ceb in term 0.
I20260927 10:51:01.866365 22858 leader_election.cc:302] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4011d3c6981048f6a022680d1ff22ceb, 8c0638ffc60542228eb762af1e347b1c; no voters:
I20260927 10:51:01.866743 22944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:51:01.866830 22944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:51:01.866870 22944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:01.868144 22944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.868467 22944 leader_election.cc:288] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [CANDIDATE]: Term 1 election: Requested vote from peers 8eb41121018641b899d6d4fe61490bf3 (127.9.39.190:46581), 8c0638ffc60542228eb762af1e347b1c (127.9.39.189:44637)
I20260927 10:51:01.869374 22731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4011d3c6981048f6a022680d1ff22ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb41121018641b899d6d4fe61490bf3" is_pre_election: true
I20260927 10:51:01.869673 22731 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4011d3c6981048f6a022680d1ff22ceb in term 0.
I20260927 10:51:01.869927 22730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4011d3c6981048f6a022680d1ff22ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb41121018641b899d6d4fe61490bf3"
I20260927 10:51:01.870039 22730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:01.871251 22730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4011d3c6981048f6a022680d1ff22ceb in term 1.
I20260927 10:51:01.871747 22806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4011d3c6981048f6a022680d1ff22ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0638ffc60542228eb762af1e347b1c"
I20260927 10:51:01.871855 22806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:01.872959 22806 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4011d3c6981048f6a022680d1ff22ceb in term 1.
I20260927 10:51:01.873301 22858 leader_election.cc:302] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4011d3c6981048f6a022680d1ff22ceb, 8c0638ffc60542228eb762af1e347b1c; no voters:
I20260927 10:51:01.876050 22944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:01.876277 22944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [term 1 LEADER]: Becoming Leader. State: Replica: 4011d3c6981048f6a022680d1ff22ceb, State: Running, Role: LEADER
I20260927 10:51:01.876591 22944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } }
I20260927 10:51:01.878364 22944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4011d3c6981048f6a022680d1ff22ceb. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.878520 22944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:01.879266 22966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:01.880314 22966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:01.887275 22806 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [term 1 FOLLOWER]: Refusing update from remote peer 4011d3c6981048f6a022680d1ff22ceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:01.887964 22964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:01.894024 22936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4011d3c6981048f6a022680d1ff22ceb. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.894135 22936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:51:01.895166 22730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [term 1 FOLLOWER]: Refusing update from remote peer 4011d3c6981048f6a022680d1ff22ceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:01.895748 22964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:01.899083 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4011d3c6981048f6a022680d1ff22ceb. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.899187 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:51:01.906481 22964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.906585 22964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:01.907701 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.907811 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:51:01.908180 22967 mvcc.cc:205] Tried to move back new op lower bound from 7333913648683188224 to 7333913648649805824. Current Snapshot: MvccSnapshot[applied={T|T < 7333913648683188224}]
I20260927 10:51:01.910068 22970 mvcc.cc:205] Tried to move back new op lower bound from 7333913648683188224 to 7333913648649805824. Current Snapshot: MvccSnapshot[applied={T|T < 7333913648683188224}]
I20260927 10:51:01.910713 22944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.910848 22944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:01.911911 22936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4011d3c6981048f6a022680d1ff22ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb41121018641b899d6d4fe61490bf3" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46581 } } peers { permanent_uuid: "8c0638ffc60542228eb762af1e347b1c" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 44637 } } peers { permanent_uuid: "4011d3c6981048f6a022680d1ff22ceb" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 45405 } } }
I20260927 10:51:01.912014 22936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:51:01.912744 22968 mvcc.cc:205] Tried to move back new op lower bound from 7333913648683188224 to 7333913648649805824. Current Snapshot: MvccSnapshot[applied={T|T < 7333913648683188224}]
I20260927 10:51:01.917191 22966 catalog_manager.cc:1347] Generated new cluster ID: a530cef2b01746329fd651088ba90516
I20260927 10:51:01.917244 22966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:51:01.965049 22966 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:51:01.966341 22966 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:51:01.996160 22966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4011d3c6981048f6a022680d1ff22ceb: Generated new TSK 0
I20260927 10:51:01.997027 22966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:51:02.797959 22929 catalog_manager.cc:1259] Loaded cluster ID: a530cef2b01746329fd651088ba90516
I20260927 10:51:02.798079 22929 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: loading cluster ID for follower catalog manager: success
I20260927 10:51:02.799873 22929 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: acquiring CA information for follower catalog manager: success
I20260927 10:51:02.800990 22929 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8eb41121018641b899d6d4fe61490bf3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 10:51:02.805364 22961 catalog_manager.cc:1259] Loaded cluster ID: a530cef2b01746329fd651088ba90516
I20260927 10:51:02.805447 22961 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: loading cluster ID for follower catalog manager: success
I20260927 10:51:02.806921 22961 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: acquiring CA information for follower catalog manager: success
I20260927 10:51:02.807928 22961 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c0638ffc60542228eb762af1e347b1c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 10:51:03.061589 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22689
I20260927 10:51:03.068194 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22764
I20260927 10:51:03.074261 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22840
2026-09-27T10:51:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3125 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:51:03 dist-test-slave-p9q6 krb5kdc[22981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:51:03 dist-test-slave-p9q6 krb5kdc[22981](info): set up 2 sockets
Sep 27 10:51:03 dist-test-slave-p9q6 krb5kdc[22981](info): commencing operation
krb5kdc: starting...
W20260927 10:51:04.114943 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:51:04 dist-test-slave-p9q6 krb5kdc[22981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:51:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:51:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:51:04.234637 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40325
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:45119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:04.342724 22994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:04.342996 22994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:04.343036 22994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:04.347373 22994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:04.347461 22994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:04.347498 22994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:04.347535 22994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:04.347569 22994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:04.355819 22994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:04.356930 22994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:04.358007 22994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:04.366608 22999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:04.366647 23002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:04.366618 23000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:04.367041 22994 server_base.cc:1034] running on GCE node
I20260927 10:51:04.367633 22994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:04.368696 22994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:04.369887 22994 hybrid_clock.cc:648] HybridClock initialized: now 1790506264369869 us; error 35 us; skew 500 ppm
Sep 27 10:51:04 dist-test-slave-p9q6 krb5kdc[22981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506264, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:04.372550 22994 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:04.373368 22994 webserver.cc:460] Webserver started at http://127.9.39.190:36189/ using document root <none> and password file <none>
I20260927 10:51:04.373924 22994 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:04.374022 22994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:04.374249 22994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:04.376308 22994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "dccf193b0cf14d4d92e149c8e75b4834"
format_stamp: "Formatted at 2026-09-27 10:51:04 on dist-test-slave-p9q6"
server_key: "94ec6412c5b6d0bf305eca39f2b3cba1"
server_key_iv: "50890008e5866e121a4e7c20562e614c"
server_key_version: "encryptionkey@0"
I20260927 10:51:04.376860 22994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "dccf193b0cf14d4d92e149c8e75b4834"
format_stamp: "Formatted at 2026-09-27 10:51:04 on dist-test-slave-p9q6"
server_key: "94ec6412c5b6d0bf305eca39f2b3cba1"
server_key_iv: "50890008e5866e121a4e7c20562e614c"
server_key_version: "encryptionkey@0"
I20260927 10:51:04.380496 22994 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260927 10:51:04.382846 23009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:04.384001 22994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:04.384137 22994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "dccf193b0cf14d4d92e149c8e75b4834"
format_stamp: "Formatted at 2026-09-27 10:51:04 on dist-test-slave-p9q6"
server_key: "94ec6412c5b6d0bf305eca39f2b3cba1"
server_key_iv: "50890008e5866e121a4e7c20562e614c"
server_key_version: "encryptionkey@0"
I20260927 10:51:04.384248 22994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:04.394860 22994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:04.397851 22994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:04.398053 22994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:04.404690 22994 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:40325
I20260927 10:51:04.404750 23061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:40325 every 8 connection(s)
I20260927 10:51:04.405690 22994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:04.408453 23062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:04.410530 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 22994
I20260927 10:51:04.410670 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:51:04.411015 9374 external_mini_cluster.cc:1428] Setting key bec64e38ef9cfa951a74e013d899e18b
I20260927 10:51:04.414510 23062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834: Bootstrap starting.
I20260927 10:51:04.416999 23062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:04.417783 23062 log.cc:824] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:04.419661 23062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834: No bootstrap required, opened a new log
Sep 27 10:51:04 dist-test-slave-p9q6 krb5kdc[22981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:04.422848 23062 raft_consensus.cc:348] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf193b0cf14d4d92e149c8e75b4834" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40325 } }
I20260927 10:51:04.423050 23062 raft_consensus.cc:374] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:04.423106 23062 raft_consensus.cc:729] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dccf193b0cf14d4d92e149c8e75b4834, State: Initialized, Role: FOLLOWER
I20260927 10:51:04.423514 23062 consensus_queue.cc:260] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf193b0cf14d4d92e149c8e75b4834" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40325 } }
I20260927 10:51:04.423676 23062 raft_consensus.cc:388] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:51:04.423750 23062 raft_consensus.cc:482] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:51:04.423854 23062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:04.424837 23062 raft_consensus.cc:504] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf193b0cf14d4d92e149c8e75b4834" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40325 } }
I20260927 10:51:04.425194 23062 leader_election.cc:302] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dccf193b0cf14d4d92e149c8e75b4834; no voters:
I20260927 10:51:04.425540 23062 leader_election.cc:288] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:51:04.425642 23067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:04.425849 23067 raft_consensus.cc:686] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [term 1 LEADER]: Becoming Leader. State: Replica: dccf193b0cf14d4d92e149c8e75b4834, State: Running, Role: LEADER
I20260927 10:51:04.426205 23067 consensus_queue.cc:237] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf193b0cf14d4d92e149c8e75b4834" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40325 } }
I20260927 10:51:04.426820 23062 sys_catalog.cc:564] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:04.426918 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:04.412312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.9.39.190:40325)
0927 10:51:04.412733 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:04.412752 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:04.412793 (+ 41us) server_negotiation.cc:407] Connection header received
0927 10:51:04.413599 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:04.413622 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:04.414010 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:04.414365 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:04.415138 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:04.416602 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:04.417568 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:04.418244 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:04.418424 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:04.421080 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:04.421084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:04.421095 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:04.421120 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:04.423520 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:04.424100 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:04.424105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:04.424111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:04.424228 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:04.424516 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:04.424518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:04.424519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:04.424874 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:04.425076 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:04.425427 (+ 351us) server_negotiation.cc:299] Negotiation successful
0927 10:51:04.425916 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
I20260927 10:51:04.428648 23069 sys_catalog.cc:455] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dccf193b0cf14d4d92e149c8e75b4834. Latest consensus state: current_term: 1 leader_uuid: "dccf193b0cf14d4d92e149c8e75b4834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf193b0cf14d4d92e149c8e75b4834" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40325 } } }
I20260927 10:51:04.428768 23069 sys_catalog.cc:458] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:04.429031 23068 sys_catalog.cc:455] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dccf193b0cf14d4d92e149c8e75b4834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf193b0cf14d4d92e149c8e75b4834" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40325 } } }
I20260927 10:51:04.429222 23068 sys_catalog.cc:458] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:04.430142 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260927 10:51:04.431738 23081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:51:04.431887 23081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:51:04.432006 23082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:04.432668 23082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:04.437615 23082 catalog_manager.cc:1347] Generated new cluster ID: 4af41796956440f197e684b84244c6ba
I20260927 10:51:04.437685 23082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:51:04.456727 23082 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:51:04.457829 23082 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:51:04.465042 23082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dccf193b0cf14d4d92e149c8e75b4834: Generated new TSK 0
I20260927 10:51:04.465696 23082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:51:04.486568 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 22994
2026-09-27T10:51:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1409 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:51:04 dist-test-slave-p9q6 krb5kdc[23089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:51:04 dist-test-slave-p9q6 krb5kdc[23089](info): set up 2 sockets
Sep 27 10:51:04 dist-test-slave-p9q6 krb5kdc[23089](info): commencing operation
krb5kdc: starting...
W20260927 10:51:05.522796 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:51:05 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:51:05.642117 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39751
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:05.750428 23102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:05.750772 23102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:05.750874 23102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:05.755046 23102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:05.755141 23102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:05.755175 23102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:05.755215 23102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:05.755245 23102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:05.760593 23102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:05.761652 23102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:05.762648 23102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:05.770823 23108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:05.770848 23107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:05.770819 23110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:05.771068 23102 server_base.cc:1034] running on GCE node
I20260927 10:51:05.771668 23102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:05.772732 23102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:05.773916 23102 hybrid_clock.cc:648] HybridClock initialized: now 1790506265773899 us; error 29 us; skew 500 ppm
Sep 27 10:51:05 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:05.776551 23102 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:05.777431 23102 webserver.cc:460] Webserver started at http://127.9.39.190:41507/ using document root <none> and password file <none>
I20260927 10:51:05.777968 23102 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:05.778053 23102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:05.778273 23102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:05.780296 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "7951fbdb097c49fc8eb550535897c5b1"
format_stamp: "Formatted at 2026-09-27 10:51:05 on dist-test-slave-p9q6"
server_key: "7bd6e68b8567e120ae87b1a0c6b9eb40"
server_key_iv: "b16db7f2a7a9cede26ff7cfcb77c5afd"
server_key_version: "encryptionkey@0"
I20260927 10:51:05.780846 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "7951fbdb097c49fc8eb550535897c5b1"
format_stamp: "Formatted at 2026-09-27 10:51:05 on dist-test-slave-p9q6"
server_key: "7bd6e68b8567e120ae87b1a0c6b9eb40"
server_key_iv: "b16db7f2a7a9cede26ff7cfcb77c5afd"
server_key_version: "encryptionkey@0"
I20260927 10:51:05.784334 23102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260927 10:51:05.786615 23117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:05.787806 23102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:05.787936 23102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "7951fbdb097c49fc8eb550535897c5b1"
format_stamp: "Formatted at 2026-09-27 10:51:05 on dist-test-slave-p9q6"
server_key: "7bd6e68b8567e120ae87b1a0c6b9eb40"
server_key_iv: "b16db7f2a7a9cede26ff7cfcb77c5afd"
server_key_version: "encryptionkey@0"
I20260927 10:51:05.788043 23102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:05.802462 23102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:05.805263 23102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:05.805467 23102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:05.812400 23102 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39751
I20260927 10:51:05.812685 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39751 every 8 connection(s)
I20260927 10:51:05.813337 23102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:05.816025 23170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:05.818145 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23102
I20260927 10:51:05.818245 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:51:05.818527 9374 external_mini_cluster.cc:1428] Setting key 51fccca1af4dcb0a84ad9b8aec93c16a
I20260927 10:51:05.821960 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap starting.
I20260927 10:51:05.824451 23170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:05.825246 23170 log.cc:824] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:05.827286 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: No bootstrap required, opened a new log
Sep 27 10:51:05 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:05.831127 23170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:05.831350 23170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:05.831441 23170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7951fbdb097c49fc8eb550535897c5b1, State: Initialized, Role: FOLLOWER
I20260927 10:51:05.831880 23170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:05.832082 23170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:51:05.832170 23170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:51:05.832280 23170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:05.832897 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:05.819984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58432 (local address 127.9.39.190:39751)
0927 10:51:05.820363 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:05.820384 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:05.820417 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:51:05.821037 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:05.821052 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:05.821403 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:05.821707 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:05.822646 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:05.823680 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:05.824837 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:05.825303 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:05.825507 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:05.828216 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:05.828221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:05.828234 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:05.828259 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:05.829920 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:05.830345 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:05.830355 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:05.830360 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:05.830475 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:05.830792 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:05.830795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:05.830797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:05.831177 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:05.831393 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:05.831695 (+ 302us) server_negotiation.cc:299] Negotiation successful
0927 10:51:05.831992 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
I20260927 10:51:05.833228 23170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:05.833559 23170 leader_election.cc:302] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7951fbdb097c49fc8eb550535897c5b1; no voters:
I20260927 10:51:05.833878 23170 leader_election.cc:288] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:51:05.834028 23175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:05.834251 23175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 LEADER]: Becoming Leader. State: Replica: 7951fbdb097c49fc8eb550535897c5b1, State: Running, Role: LEADER
I20260927 10:51:05.834548 23175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:05.835140 23170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:05.835711 23175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7951fbdb097c49fc8eb550535897c5b1. Latest consensus state: current_term: 1 leader_uuid: "7951fbdb097c49fc8eb550535897c5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } } }
I20260927 10:51:05.835825 23175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:05.836201 23181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:05.836153 23176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7951fbdb097c49fc8eb550535897c5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } } }
I20260927 10:51:05.836282 23176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:05.838876 23181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:05.843678 23181 catalog_manager.cc:1347] Generated new cluster ID: 4fba3c3ce577486db9445971c056f4aa
I20260927 10:51:05.843732 23181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:51:05.850570 23181 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:51:05.851429 23181 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:51:05.861508 23181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Generated new TSK 0
I20260927 10:51:05.862149 23181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:51:05.890199 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39751
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:05.999648 23196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:05.999945 23196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:06.000020 23196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:06.003981 23196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:06.004068 23196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:06.004160 23196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:51:06.009403 23196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:06.010489 23196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:06.011559 23196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:06.021209 23201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:06.021255 23196 server_base.cc:1034] running on GCE node
W20260927 10:51:06.021234 23204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:06.021219 23202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:06.021802 23196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:06.022455 23196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:06.023640 23196 hybrid_clock.cc:648] HybridClock initialized: now 1790506266023617 us; error 34 us; skew 500 ppm
Sep 27 10:51:06 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:06.026283 23196 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:51:06.027133 23196 webserver.cc:460] Webserver started at http://127.9.39.129:46703/ using document root <none> and password file <none>
I20260927 10:51:06.027649 23196 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:06.027707 23196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:06.027864 23196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:06.029767 23196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "49a0f1776bd34ade9498e5e5874b146b"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "ce5ed4da0c6fed0f5de2772a12d444da"
server_key_iv: "1f868ab71ac099a0a8c6ad677109f8ae"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.030264 23196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "49a0f1776bd34ade9498e5e5874b146b"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "ce5ed4da0c6fed0f5de2772a12d444da"
server_key_iv: "1f868ab71ac099a0a8c6ad677109f8ae"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.033910 23196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:51:06.036360 23211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.037648 23196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:51:06.037774 23196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "49a0f1776bd34ade9498e5e5874b146b"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "ce5ed4da0c6fed0f5de2772a12d444da"
server_key_iv: "1f868ab71ac099a0a8c6ad677109f8ae"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.037879 23196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:06.051421 23196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:06.054492 23196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:06.054726 23196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:06.055301 23196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:06.056102 23196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:06.056170 23196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.056237 23196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:06.056288 23196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.065037 23196 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:46581
I20260927 10:51:06.065092 23324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:46581 every 8 connection(s)
I20260927 10:51:06.065969 23196 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:51:06.066794 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23196
I20260927 10:51:06.066933 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:51:06.067209 9374 external_mini_cluster.cc:1428] Setting key e474fef02645c72577c85d0038fe6ef0
Sep 27 10:51:06 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:51:06.079908 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.069368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:42135 (local address 127.9.39.190:39751)
0927 10:51:06.069520 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.069526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.069542 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:51:06.070115 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.070118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.070172 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.070249 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:06.071422 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.071964 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.073102 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.073352 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.073465 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.076151 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:06.076155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:06.076157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:06.076188 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:06.077714 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.078249 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.078252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.078253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.078326 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.078678 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.078684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.078686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.078963 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:06.079072 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.079669 (+ 597us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.079779 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 10:51:06.080974 23327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.068282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:42135)
0927 10:51:06.068980 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.069032 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.069912 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.070429 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.070436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.070805 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:06.071270 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.071283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.072090 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.072094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.072972 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.072981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.073505 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.073509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.073698 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.074316 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:06.074331 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:06.075986 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:06.077817 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:06.077822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:06.077834 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:06.078158 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:06.078405 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:06.078407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:06.078409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:06.078568 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:06.079081 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:06.079090 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:06.079408 (+ 318us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.079634 (+ 226us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.080004 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":136,"threads_started":1}
I20260927 10:51:06.082192 23325 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39751
I20260927 10:51:06.082450 23325 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:06.083108 23325 heartbeater.cc:511] Master 127.9.39.190:39751 requested a full tablet report, sending...
I20260927 10:51:06.084719 23134 ts_manager.cc:194] Registered new tserver with Master: 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581)
I20260927 10:51:06.085894 23134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:42135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:51:06.094066 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39751
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:06.203521 23330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:06.203792 23330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:06.203866 23330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:06.207706 23330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:06.207780 23330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:06.207861 23330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:51:06.212882 23330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:06.213871 23330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:06.214864 23330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:06.224061 23336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:06.224050 23338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:06.224123 23335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:06.224217 23330 server_base.cc:1034] running on GCE node
I20260927 10:51:06.224747 23330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:06.225347 23330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:06.226507 23330 hybrid_clock.cc:648] HybridClock initialized: now 1790506266226492 us; error 25 us; skew 500 ppm
Sep 27 10:51:06 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:06.229056 23330 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:51:06.229888 23330 webserver.cc:460] Webserver started at http://127.9.39.130:36973/ using document root <none> and password file <none>
I20260927 10:51:06.230384 23330 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:06.230441 23330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:06.230597 23330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:06.232570 23330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "3496156c65ef4f96ae0afe105be28699"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "b56841a198f76a0e5d5bcf7bb49af3e0"
server_key_iv: "efec9393ebef3d1ad22b68f302e14e45"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.233069 23330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "3496156c65ef4f96ae0afe105be28699"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "b56841a198f76a0e5d5bcf7bb49af3e0"
server_key_iv: "efec9393ebef3d1ad22b68f302e14e45"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.236816 23330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 10:51:06.239121 23345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.240397 23330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:06.240542 23330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3496156c65ef4f96ae0afe105be28699"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "b56841a198f76a0e5d5bcf7bb49af3e0"
server_key_iv: "efec9393ebef3d1ad22b68f302e14e45"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.240659 23330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:06.256150 23330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:06.259024 23330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:06.259231 23330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:06.259857 23330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:06.260711 23330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:06.260793 23330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.260864 23330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:06.260914 23330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.270033 23330 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:35401
I20260927 10:51:06.270045 23458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:35401 every 8 connection(s)
I20260927 10:51:06.270985 23330 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
Sep 27 10:51:06 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:06.280278 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23330
I20260927 10:51:06.280423 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:51:06.280762 9374 external_mini_cluster.cc:1428] Setting key 9f426b8bb2dd40247771e5519eb0d9ca
I20260927 10:51:06.284204 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.272754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:45161 (local address 127.9.39.190:39751)
0927 10:51:06.272888 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.272891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.273850 (+ 959us) server_negotiation.cc:407] Connection header received
0927 10:51:06.274782 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.274785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.274852 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.274957 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:06.276120 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.276686 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.277832 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.278063 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.278196 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.280676 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:06.280680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:06.280683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:06.280713 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:06.282147 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.282654 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.282659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.282660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.282786 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.283132 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.283137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.283139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.283315 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:06.283420 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.283914 (+ 494us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.284041 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:51:06.284948 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.273029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:45161)
0927 10:51:06.273692 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.273737 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.274537 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.275138 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.275145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.275519 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:06.275965 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.275979 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.276827 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.276831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.277689 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.277698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.278239 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.278245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.278418 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.279019 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:06.279047 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:06.280506 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:06.282266 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:06.282272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:06.282284 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:06.282541 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:06.282878 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:06.282881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:06.282883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:06.283019 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:06.283418 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:06.283425 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:06.283655 (+ 230us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.283841 (+ 186us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.284166 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":109,"threads_started":1}
I20260927 10:51:06.286175 23459 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39751
I20260927 10:51:06.286427 23459 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:06.286937 23459 heartbeater.cc:511] Master 127.9.39.190:39751 requested a full tablet report, sending...
I20260927 10:51:06.287995 23134 ts_manager.cc:194] Registered new tserver with Master: 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
I20260927 10:51:06.288782 23134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:45161
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:51:06.307696 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39751
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:06.417585 23464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:06.417819 23464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:06.417860 23464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:06.422089 23464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:06.422143 23464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:06.422199 23464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:51:06.427340 23464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:06.428323 23464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:06.429234 23464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:06.438637 23472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:06.438655 23470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:06.438656 23469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:06.439036 23464 server_base.cc:1034] running on GCE node
I20260927 10:51:06.439396 23464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:06.440006 23464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:06.441207 23464 hybrid_clock.cc:648] HybridClock initialized: now 1790506266441180 us; error 46 us; skew 500 ppm
Sep 27 10:51:06 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:06.444023 23464 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:51:06.444947 23464 webserver.cc:460] Webserver started at http://127.9.39.131:36191/ using document root <none> and password file <none>
I20260927 10:51:06.445513 23464 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:06.445600 23464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:06.445811 23464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:06.447805 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "7db46a991a64e99b022accb4ce3e99a5"
server_key_iv: "3b0766483d4fe69e4e7b790cce2716d7"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.448341 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "7db46a991a64e99b022accb4ce3e99a5"
server_key_iv: "3b0766483d4fe69e4e7b790cce2716d7"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.452124 23464 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:51:06.454602 23479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.455852 23464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:06.456000 23464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "7db46a991a64e99b022accb4ce3e99a5"
server_key_iv: "3b0766483d4fe69e4e7b790cce2716d7"
server_key_version: "encryptionkey@0"
I20260927 10:51:06.456113 23464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:06.476647 23464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:06.480114 23464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:06.480319 23464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:06.480935 23464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:06.481747 23464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:06.481819 23464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.481894 23464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:06.481959 23464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:06.490865 23464 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:41033
I20260927 10:51:06.490984 23592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:41033 every 8 connection(s)
I20260927 10:51:06.491801 23464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:51:06.493506 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23464
I20260927 10:51:06.493636 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:51:06.493899 9374 external_mini_cluster.cc:1428] Setting key 579e40b3304ec3b12800e69ee414b38f
Sep 27 10:51:06 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:06.504550 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.493619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:57221 (local address 127.9.39.190:39751)
0927 10:51:06.493736 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.493741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.495149 (+ 1408us) server_negotiation.cc:407] Connection header received
0927 10:51:06.496032 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.496036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.496091 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.496171 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:06.497259 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.497777 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.498917 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.499136 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.499229 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.501498 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:06.501501 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:06.501503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:06.501530 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:06.502787 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.503334 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.503336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.503338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.503416 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.503700 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.503703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.503704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.503862 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:06.503935 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.504350 (+ 415us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.504437 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 10:51:06.505396 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.494065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:57221)
0927 10:51:06.495006 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.495062 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.495832 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.496331 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.496338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.496671 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:06.497137 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.497149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.497876 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.497878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.498813 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.498823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.499265 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.499268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.499415 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.500000 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:06.500015 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:06.501356 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:06.502884 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:06.502889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:06.502899 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:06.503158 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:06.503500 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:06.503502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:06.503504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:06.503623 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:06.503952 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:06.503956 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:06.504159 (+ 203us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.504305 (+ 146us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.504533 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":125,"threads_started":1}
I20260927 10:51:06.506553 23593 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39751
I20260927 10:51:06.506803 23593 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:06.507263 23593 heartbeater.cc:511] Master 127.9.39.190:39751 requested a full tablet report, sending...
I20260927 10:51:06.508225 23134 ts_manager.cc:194] Registered new tserver with Master: 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033)
I20260927 10:51:06.508769 23134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:57221
I20260927 10:51:06.519349 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:51:06.528074 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.521242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58438 (local address 127.9.39.190:39751)
0927 10:51:06.521361 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.521365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.521544 (+ 179us) server_negotiation.cc:407] Connection header received
0927 10:51:06.521630 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.521632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.521667 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.521731 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:06.522411 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.523018 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.523782 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.523990 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.524062 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.525279 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:06.525282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:06.525284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:06.525305 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:06.526799 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.527137 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.527139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.527140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.527209 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:06.527419 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:06.527421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:06.527422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:06.527605 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:06.527705 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.527868 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.527960 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:51:06.530768 23134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:51:06.533677 23134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:51:06.547780 23605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.541911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:41033 (local address 127.0.0.1:38432)
0927 10:51:06.543202 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.543213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.543295 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.543727 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.543730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.543750 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:06.543879 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.543888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.545198 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.545202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.546242 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.546251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.547200 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.547204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.547320 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.547359 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.547486 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.547601 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1232,"mutex_wait_us":5,"thread_start_us":101,"threads_started":1}
I20260927 10:51:06.547794 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.542599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.9.39.131:41033)
0927 10:51:06.543059 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.543072 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.543320 (+ 248us) server_negotiation.cc:407] Connection header received
0927 10:51:06.543397 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.543401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.543567 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.543701 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:06.544005 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.545047 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.546367 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.547081 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.547188 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.547325 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.547528 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.547630 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":184,"threads_started":1}
I20260927 10:51:06.548417 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.541841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46581 (local address 127.0.0.1:44766)
0927 10:51:06.542669 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.542741 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.542856 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.543546 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.543552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.543575 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:06.543706 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.543712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.545359 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.545363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.546570 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.546580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.547817 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.547822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.548064 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.548096 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.548191 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.548282 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":50,"threads_started":1}
I20260927 10:51:06.548812 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.542592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.9.39.130:35401)
0927 10:51:06.543120 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.543131 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.543654 (+ 523us) server_negotiation.cc:407] Connection header received
0927 10:51:06.543732 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.543740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.543999 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.544145 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:06.544361 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.545499 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.546881 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.547622 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.547717 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.547856 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.548558 (+ 702us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.548668 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":140,"threads_started":1}
I20260927 10:51:06.548847 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.541944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.9.39.129:46581)
0927 10:51:06.543077 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.543090 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.543108 (+ 18us) server_negotiation.cc:407] Connection header received
0927 10:51:06.543168 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.543172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.543374 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.543537 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:06.543838 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.545215 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.546720 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.547516 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.547617 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.547752 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.548610 (+ 858us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.548705 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":92,"threads_started":1}
I20260927 10:51:06.548470 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.542177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35401 (local address 127.0.0.1:50028)
0927 10:51:06.543438 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.543447 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.543541 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.544134 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.544137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.544155 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:06.544242 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.544248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.545636 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.545639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.546773 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.546781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.547931 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.547935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.548110 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.548150 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.548239 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.548325 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":70,"thread_start_us":89,"threads_started":1}
I20260927 10:51:06.551047 23527 tablet_service.cc:1467] Processing CreateTablet for tablet 86dd74976d1840b38c5da22972ce9db3 (DEFAULT_TABLE table=test-table [id=8b4e3578c01d4e54983e6ff8a3dd34e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:06.552043 23527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86dd74976d1840b38c5da22972ce9db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:06.551605 23259 tablet_service.cc:1467] Processing CreateTablet for tablet 86dd74976d1840b38c5da22972ce9db3 (DEFAULT_TABLE table=test-table [id=8b4e3578c01d4e54983e6ff8a3dd34e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:06.551606 23393 tablet_service.cc:1467] Processing CreateTablet for tablet 86dd74976d1840b38c5da22972ce9db3 (DEFAULT_TABLE table=test-table [id=8b4e3578c01d4e54983e6ff8a3dd34e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:06.552626 23393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86dd74976d1840b38c5da22972ce9db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:06.552626 23259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86dd74976d1840b38c5da22972ce9db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:06.557879 23610 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Bootstrap starting.
I20260927 10:51:06.559443 23611 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Bootstrap starting.
I20260927 10:51:06.560297 23612 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap starting.
I20260927 10:51:06.560797 23610 tablet_bootstrap.cc:654] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:06.561739 23610 log.cc:824] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:06.562192 23612 tablet_bootstrap.cc:654] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:06.562220 23611 tablet_bootstrap.cc:654] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:06.563097 23611 log.cc:824] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:06.563378 23612 log.cc:824] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:06.564311 23610 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: No bootstrap required, opened a new log
I20260927 10:51:06.564509 23610 ts_tablet_manager.cc:1397] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260927 10:51:06.564982 23611 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: No bootstrap required, opened a new log
I20260927 10:51:06.565174 23612 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: No bootstrap required, opened a new log
I20260927 10:51:06.565181 23611 ts_tablet_manager.cc:1397] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:51:06.565336 23612 ts_tablet_manager.cc:1397] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260927 10:51:06.567508 23610 raft_consensus.cc:348] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.567739 23610 raft_consensus.cc:374] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:06.567790 23610 raft_consensus.cc:729] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fa368bc0c04bf8b9d5ff48eda4e0f2, State: Initialized, Role: FOLLOWER
I20260927 10:51:06.568395 23610 consensus_queue.cc:260] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.569049 23593 heartbeater.cc:503] Master 127.9.39.190:39751 was elected leader, sending a full tablet report...
I20260927 10:51:06.569031 23612 raft_consensus.cc:348] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.569032 23611 raft_consensus.cc:348] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.569303 23611 raft_consensus.cc:374] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:06.569303 23612 raft_consensus.cc:374] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:06.569351 23611 raft_consensus.cc:729] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3496156c65ef4f96ae0afe105be28699, State: Initialized, Role: FOLLOWER
I20260927 10:51:06.569360 23612 raft_consensus.cc:729] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0f1776bd34ade9498e5e5874b146b, State: Initialized, Role: FOLLOWER
I20260927 10:51:06.569751 23612 consensus_queue.cc:260] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.569751 23611 consensus_queue.cc:260] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.570225 23610 ts_tablet_manager.cc:1428] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 10:51:06.570662 23611 ts_tablet_manager.cc:1428] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:51:06.570662 23612 ts_tablet_manager.cc:1428] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260927 10:51:06.571017 23459 heartbeater.cc:503] Master 127.9.39.190:39751 was elected leader, sending a full tablet report...
I20260927 10:51:06.571029 23325 heartbeater.cc:503] Master 127.9.39.190:39751 was elected leader, sending a full tablet report...
W20260927 10:51:06.742861 23594 tablet.cc:2369] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:51:06.772198 23460 tablet.cc:2369] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:06.782979 23618 raft_consensus.cc:482] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:06.783227 23618 raft_consensus.cc:504] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.784389 23618 leader_election.cc:288] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581), 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033)
I20260927 10:51:06.788190 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.784615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46581 (local address 127.9.39.130:53945)
0927 10:51:06.784735 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.784751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.784844 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.785202 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.785206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.785227 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:06.785361 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.785370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.786177 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.786180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.787024 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.787033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.787712 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.787716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.787842 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.787872 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.787962 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.788050 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 10:51:06.788216 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.784770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:53945 (local address 127.9.39.129:46581)
0927 10:51:06.784891 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.784896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.784918 (+ 22us) server_negotiation.cc:407] Connection header received
0927 10:51:06.785018 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.785022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.785072 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.785165 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:06.785479 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.786057 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.787147 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.787596 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.787688 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.787729 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.788004 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.788078 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:51:06.788363 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.784621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:41033 (local address 127.9.39.130:56619)
0927 10:51:06.785072 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.785085 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:06.785181 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:06.785415 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:06.785417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:06.785461 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:06.785566 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.785572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.786298 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.786300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.787203 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:06.787213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.787891 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.787893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:06.788038 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.788060 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:51:06.788134 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 10:51:06.788194 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":169,"thread_start_us":103,"threads_started":1}
I20260927 10:51:06.788379 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.784735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:56619 (local address 127.9.39.131:41033)
0927 10:51:06.784858 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.784863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.785176 (+ 313us) server_negotiation.cc:407] Connection header received
0927 10:51:06.785263 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.785265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.785316 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.785399 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:06.785671 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.786193 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.787322 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.787766 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.787889 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.787979 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.788168 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.788253 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 10:51:06.788836 23279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dd74976d1840b38c5da22972ce9db3" candidate_uuid: "3496156c65ef4f96ae0afe105be28699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a0f1776bd34ade9498e5e5874b146b" is_pre_election: true
I20260927 10:51:06.788898 23547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dd74976d1840b38c5da22972ce9db3" candidate_uuid: "3496156c65ef4f96ae0afe105be28699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" is_pre_election: true
I20260927 10:51:06.789157 23279 raft_consensus.cc:2454] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3496156c65ef4f96ae0afe105be28699 in term 0.
I20260927 10:51:06.789188 23547 raft_consensus.cc:2454] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3496156c65ef4f96ae0afe105be28699 in term 0.
I20260927 10:51:06.789625 23346 leader_election.cc:302] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 3496156c65ef4f96ae0afe105be28699; no voters:
I20260927 10:51:06.789875 23618 raft_consensus.cc:2781] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:51:06.789956 23618 raft_consensus.cc:482] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:51:06.789999 23618 raft_consensus.cc:3037] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:06.791044 23618 raft_consensus.cc:504] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.791365 23618 leader_election.cc:288] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [CANDIDATE]: Term 1 election: Requested vote from peers 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581), 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033)
I20260927 10:51:06.791702 23547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dd74976d1840b38c5da22972ce9db3" candidate_uuid: "3496156c65ef4f96ae0afe105be28699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
I20260927 10:51:06.791805 23547 raft_consensus.cc:3037] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:06.792732 23547 raft_consensus.cc:2454] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3496156c65ef4f96ae0afe105be28699 in term 1.
I20260927 10:51:06.793009 23346 leader_election.cc:302] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 3496156c65ef4f96ae0afe105be28699; no voters:
I20260927 10:51:06.793004 23279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dd74976d1840b38c5da22972ce9db3" candidate_uuid: "3496156c65ef4f96ae0afe105be28699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a0f1776bd34ade9498e5e5874b146b"
I20260927 10:51:06.793143 23279 raft_consensus.cc:3037] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:06.793193 23618 raft_consensus.cc:2781] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:06.793426 23618 raft_consensus.cc:686] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 1 LEADER]: Becoming Leader. State: Replica: 3496156c65ef4f96ae0afe105be28699, State: Running, Role: LEADER
I20260927 10:51:06.793782 23618 consensus_queue.cc:237] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:06.794219 23279 raft_consensus.cc:2454] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3496156c65ef4f96ae0afe105be28699 in term 1.
I20260927 10:51:06.796968 23133 catalog_manager.cc:5495] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3496156c65ef4f96ae0afe105be28699 (127.9.39.130). New cstate: current_term: 1 leader_uuid: "3496156c65ef4f96ae0afe105be28699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } health_report { overall_health: HEALTHY } } }
W20260927 10:51:06.817389 23326 tablet.cc:2369] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:06.836530 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.833038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.9.39.130:35401)
0927 10:51:06.833165 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:06.833168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:06.833181 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:51:06.833253 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:06.833255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:06.833308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:06.833407 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:51:06.833624 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.834113 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.834964 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:06.835165 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:06.835242 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:06.835590 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:51:06.836077 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:51:06.836207 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:06.836266 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 10:51:06.836324 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:51:06.842341 23279 raft_consensus.cc:1261] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Refusing update from remote peer 3496156c65ef4f96ae0afe105be28699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:06.842341 23547 raft_consensus.cc:1261] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Refusing update from remote peer 3496156c65ef4f96ae0afe105be28699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:06.843180 23620 consensus_queue.cc:1035] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:06.843411 23618 consensus_queue.cc:1035] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:06.849759 23625 mvcc.cc:205] Tried to move back new op lower bound from 7333913668979003392 to 7333913668790165504. Current Snapshot: MvccSnapshot[applied={T|T < 7333913668979003392}]
I20260927 10:51:06.849892 23623 mvcc.cc:205] Tried to move back new op lower bound from 7333913668979003392 to 7333913668790165504. Current Snapshot: MvccSnapshot[applied={T|T < 7333913668979003392}]
I20260927 10:51:06.851784 23624 mvcc.cc:205] Tried to move back new op lower bound from 7333913668979003392 to 7333913668790165504. Current Snapshot: MvccSnapshot[applied={T|T < 7333913668979003392}]
I20260927 10:51:06.858511 23132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:06.858652 23132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:06.860958 23132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Sending DeleteTablet for 3 replicas of tablet 86dd74976d1840b38c5da22972ce9db3
I20260927 10:51:06.861672 23259 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dd74976d1840b38c5da22972ce9db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:06 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:44766
I20260927 10:51:06.861792 23527 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dd74976d1840b38c5da22972ce9db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:06 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:38432
I20260927 10:51:06.862118 23632 tablet_replica.cc:330] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: stopping tablet replica
I20260927 10:51:06.862149 23393 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dd74976d1840b38c5da22972ce9db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:06 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:50028
I20260927 10:51:06.862421 23632 raft_consensus.cc:2229] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:06.862743 23632 raft_consensus.cc:2258] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:06.862808 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23196
I20260927 10:51:06.863746 23633 tablet_replica.cc:330] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: stopping tablet replica
I20260927 10:51:06.864014 23633 raft_consensus.cc:2229] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:06.862771 23634 tablet_replica.cc:330] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: stopping tablet replica
I20260927 10:51:06.864324 23633 raft_consensus.cc:2258] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:06.864459 23634 raft_consensus.cc:2229] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:51:06.864799 23634 raft_consensus.cc:2258] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:06.865722 23633 ts_tablet_manager.cc:1905] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:06.865996 23634 ts_tablet_manager.cc:1905] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:06.868388 23633 ts_tablet_manager.cc:1918] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:06.868505 23633 log.cc:1196] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/86dd74976d1840b38c5da22972ce9db3
I20260927 10:51:06.868548 23634 ts_tablet_manager.cc:1918] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:06.868643 23634 log.cc:1196] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/86dd74976d1840b38c5da22972ce9db3
I20260927 10:51:06.868854 23633 ts_tablet_manager.cc:1939] T 86dd74976d1840b38c5da22972ce9db3 P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Deleting consensus metadata
I20260927 10:51:06.868932 23634 ts_tablet_manager.cc:1939] T 86dd74976d1840b38c5da22972ce9db3 P 3496156c65ef4f96ae0afe105be28699: Deleting consensus metadata
I20260927 10:51:06.870015 23120 catalog_manager.cc:4841] TS 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401): tablet 86dd74976d1840b38c5da22972ce9db3 (table test-table [id=8b4e3578c01d4e54983e6ff8a3dd34e5]) successfully deleted
I20260927 10:51:06.870147 23118 catalog_manager.cc:4841] TS 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033): tablet 86dd74976d1840b38c5da22972ce9db3 (table test-table [id=8b4e3578c01d4e54983e6ff8a3dd34e5]) successfully deleted
W20260927 10:51:06.870515 23121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:46581 (error 108)
I20260927 10:51:06.870754 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23330
I20260927 10:51:06.872182 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:06.871823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46581 (local address 127.0.0.1:44772)
0927 10:51:06.871987 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:06.872081 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:46581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260927 10:51:06.872349 23121 catalog_manager.cc:4584] TS 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86dd74976d1840b38c5da22972ce9db3: Network error: Client connection negotiation failed: client connection to 127.9.39.129:46581: connect: Connection refused (error 111)
I20260927 10:51:06.876804 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23464
I20260927 10:51:06.882376 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23102
I20260927 10:51:06.888478 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39751
--webserver_interface=127.9.39.190
--webserver_port=41507
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:06.995975 23636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:06.996236 23636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:06.996284 23636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:07.000247 23636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:07.000296 23636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:07.000315 23636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:07.000331 23636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:07.000346 23636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:07.005578 23636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=41507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:07.006564 23636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:07.007578 23636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:07.020134 23645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.020143 23643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.020331 23642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.020521 23636 server_base.cc:1034] running on GCE node
I20260927 10:51:07.021122 23636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:07.022212 23636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:07.023402 23636 hybrid_clock.cc:648] HybridClock initialized: now 1790506267023390 us; error 32 us; skew 500 ppm
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:07.025993 23636 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:07.026942 23636 webserver.cc:460] Webserver started at http://127.9.39.190:41507/ using document root <none> and password file <none>
I20260927 10:51:07.027527 23636 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:07.027629 23636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:07.031358 23636 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 10:51:07.033365 23652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.034614 23636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:07.034780 23636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "7951fbdb097c49fc8eb550535897c5b1"
format_stamp: "Formatted at 2026-09-27 10:51:05 on dist-test-slave-p9q6"
server_key: "7bd6e68b8567e120ae87b1a0c6b9eb40"
server_key_iv: "b16db7f2a7a9cede26ff7cfcb77c5afd"
server_key_version: "encryptionkey@0"
I20260927 10:51:07.035367 23636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:07.052325 23636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:07.055178 23636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:07.055379 23636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:07.062372 23636 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39751
I20260927 10:51:07.062677 23704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39751 every 8 connection(s)
I20260927 10:51:07.063594 23636 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:07.064570 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23636
I20260927 10:51:07.065017 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:46581
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:39751
--webserver_port=46703
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:07.068116 23705 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:51:07.070003 23705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap starting.
I20260927 10:51:07.086969 23705 log.cc:824] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:07.095724 23705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:51:07.096089 23705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap complete.
I20260927 10:51:07.100020 23705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:07.100464 23705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7951fbdb097c49fc8eb550535897c5b1, State: Initialized, Role: FOLLOWER
I20260927 10:51:07.100979 23705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:07.101123 23705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:51:07.101217 23705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:51:07.101351 23705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 1 FOLLOWER]: Advancing to term 2
I20260927 10:51:07.103205 23705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:07.103533 23705 leader_election.cc:302] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7951fbdb097c49fc8eb550535897c5b1; no voters:
I20260927 10:51:07.103890 23705 leader_election.cc:288] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 10:51:07.103965 23709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 10:51:07.104153 23709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 LEADER]: Becoming Leader. State: Replica: 7951fbdb097c49fc8eb550535897c5b1, State: Running, Role: LEADER
I20260927 10:51:07.104461 23709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:07.105204 23705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:07.105487 23711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7951fbdb097c49fc8eb550535897c5b1. Latest consensus state: current_term: 2 leader_uuid: "7951fbdb097c49fc8eb550535897c5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } } }
I20260927 10:51:07.105609 23711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:07.105799 23710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7951fbdb097c49fc8eb550535897c5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } } }
I20260927 10:51:07.105891 23710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:07.105965 23713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:07.110683 23713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:07.111328 23713 catalog_manager.cc:1259] Loaded cluster ID: 4fba3c3ce577486db9445971c056f4aa
I20260927 10:51:07.111405 23713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:51:07.112804 23713 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 10:51:07.117600 23713 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 10:51:07.122875 23713 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:51:07.123594 23713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Loaded TSK: 0
I20260927 10:51:07.124718 23713 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 10:51:07.178580 23707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:07.178928 23707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:07.179021 23707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:07.183094 23707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:07.183169 23707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:07.183255 23707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:51:07.188417 23707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:46581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=46703
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:07.189427 23707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:07.190496 23707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:07.200062 23733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.200062 23732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.200194 23735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.200490 23707 server_base.cc:1034] running on GCE node
I20260927 10:51:07.200853 23707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:07.201462 23707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:07.202642 23707 hybrid_clock.cc:648] HybridClock initialized: now 1790506267202610 us; error 50 us; skew 500 ppm
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:07.205395 23707 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:51:07.206350 23707 webserver.cc:460] Webserver started at http://127.9.39.129:46703/ using document root <none> and password file <none>
I20260927 10:51:07.206993 23707 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:07.207099 23707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:07.210995 23707 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:51:07.213075 23742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.214344 23707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:07.214437 23707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "49a0f1776bd34ade9498e5e5874b146b"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "ce5ed4da0c6fed0f5de2772a12d444da"
server_key_iv: "1f868ab71ac099a0a8c6ad677109f8ae"
server_key_version: "encryptionkey@0"
I20260927 10:51:07.215042 23707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:07.226799 23707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:07.229782 23707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:07.229978 23707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:07.230574 23707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:07.231688 23749 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 10:51:07.235636 23707 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 10:51:07.235697 23707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260927 10:51:07.235761 23707 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 10:51:07.237358 23707 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 10:51:07.237411 23707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260927 10:51:07.237656 23749 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap starting.
I20260927 10:51:07.248550 23707 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:46581
I20260927 10:51:07.249832 23707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:51:07.250624 23856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:46581 every 8 connection(s)
I20260927 10:51:07.251156 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23707
I20260927 10:51:07.251598 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:35401
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:39751
--webserver_port=36973
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:07.259548 23749 log.cc:824] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Log is configured to *not* fsync() on all Append() calls
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:07.267730 23749 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:51:07.268143 23749 tablet_bootstrap.cc:492] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap complete.
I20260927 10:51:07.269335 23749 ts_tablet_manager.cc:1397] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Time spent bootstrapping tablet: real 0.032s user 0.028s sys 0.000s
I20260927 10:51:07.271957 23749 raft_consensus.cc:348] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:07.272457 23749 raft_consensus.cc:729] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0f1776bd34ade9498e5e5874b146b, State: Initialized, Role: FOLLOWER
I20260927 10:51:07.273000 23749 consensus_queue.cc:260] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } } peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } }
I20260927 10:51:07.273267 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.255573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:38063 (local address 127.9.39.190:39751)
0927 10:51:07.257644 (+ 2071us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.257661 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.257698 (+ 37us) server_negotiation.cc:407] Connection header received
0927 10:51:07.257959 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.257972 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.258322 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.258637 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:07.260467 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.261917 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.263877 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.264541 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.264706 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.267756 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:07.267763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:07.267779 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:07.267810 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:07.269830 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.270458 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.270464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.270469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.270612 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.271008 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.271012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.271014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.271422 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:07.271547 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.272133 (+ 586us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.272411 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1925,"thread_start_us":139,"threads_started":1}
I20260927 10:51:07.273934 23749 ts_tablet_manager.cc:1428] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260927 10:51:07.274092 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.254363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:38063)
0927 10:51:07.256171 (+ 1808us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.256231 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.257209 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.259089 (+ 1880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.259098 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.259552 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:07.260272 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.260295 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.262107 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.262111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.263721 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.263738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.264690 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.264694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.264895 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.265639 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:07.265658 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:07.267544 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:07.269950 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.269956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.269969 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.270315 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.270729 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.270732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.270734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.270901 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.271543 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:07.271548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:07.271806 (+ 258us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.271993 (+ 187us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.272308 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1585,"thread_start_us":128,"threads_started":1}
I20260927 10:51:07.276248 23857 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39751
I20260927 10:51:07.276510 23857 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:07.276973 23857 heartbeater.cc:511] Master 127.9.39.190:39751 requested a full tablet report, sending...
I20260927 10:51:07.279201 23669 ts_manager.cc:194] Registered new tserver with Master: 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581)
I20260927 10:51:07.282442 23669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:38063
I20260927 10:51:07.284852 23857 heartbeater.cc:503] Master 127.9.39.190:39751 was elected leader, sending a full tablet report...
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:51:07.292574 23864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.281806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46581 (local address 127.0.0.1:44788)
0927 10:51:07.282491 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.282536 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.282740 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.283341 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.283347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.283736 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:07.283855 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.283863 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.285120 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.285124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.286045 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.286056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.286911 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.286914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.287036 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.287544 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:07.287559 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:07.288998 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:07.291013 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.291021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.291025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.291234 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.291531 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.291534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.291535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.291608 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.292086 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:07.292090 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:07.292180 (+ 90us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.292302 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.292452 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":103,"threads_started":1}
I20260927 10:51:07.292727 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.281880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.9.39.129:46581)
0927 10:51:07.282227 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.282237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.282616 (+ 379us) server_negotiation.cc:407] Connection header received
0927 10:51:07.282900 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.282906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.283096 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.283251 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:07.283981 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.284989 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.286182 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.286656 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.286760 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.289125 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:07.289134 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:07.289145 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:07.289169 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:07.290893 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.291340 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.291348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.291350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.291440 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.291687 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.291693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.291696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.291978 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:07.292106 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.292406 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.292568 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":106,"threads_started":1}
I20260927 10:51:07.293576 23782 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dd74976d1840b38c5da22972ce9db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:06 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:44788
I20260927 10:51:07.294402 23866 tablet_replica.cc:330] stopping tablet replica
I20260927 10:51:07.294719 23866 raft_consensus.cc:2229] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:07.294976 23866 raft_consensus.cc:2258] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:07.295992 23866 ts_tablet_manager.cc:1905] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:07.300216 23866 ts_tablet_manager.cc:1918] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:07.300310 23866 log.cc:1196] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/86dd74976d1840b38c5da22972ce9db3
I20260927 10:51:07.300575 23866 ts_tablet_manager.cc:1939] T 86dd74976d1840b38c5da22972ce9db3 P 49a0f1776bd34ade9498e5e5874b146b: Deleting consensus metadata
I20260927 10:51:07.302320 23656 catalog_manager.cc:4841] TS 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581): tablet 86dd74976d1840b38c5da22972ce9db3 (table test-table [id=8b4e3578c01d4e54983e6ff8a3dd34e5]) successfully deleted
W20260927 10:51:07.370945 23859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:07.371186 23859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:07.371228 23859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:07.375175 23859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:07.375223 23859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:07.375314 23859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:51:07.380474 23859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:35401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=36973
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:07.381428 23859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:07.382411 23859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:07.391747 23871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.391764 23872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.391791 23874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.391901 23859 server_base.cc:1034] running on GCE node
I20260927 10:51:07.392439 23859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:07.393052 23859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:07.394207 23859 hybrid_clock.cc:648] HybridClock initialized: now 1790506267394174 us; error 45 us; skew 500 ppm
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:07.396981 23859 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:51:07.397760 23859 webserver.cc:460] Webserver started at http://127.9.39.130:36973/ using document root <none> and password file <none>
I20260927 10:51:07.398264 23859 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:07.398356 23859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:07.402060 23859 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:07.404063 23881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.405354 23859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:51:07.405491 23859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3496156c65ef4f96ae0afe105be28699"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "b56841a198f76a0e5d5bcf7bb49af3e0"
server_key_iv: "efec9393ebef3d1ad22b68f302e14e45"
server_key_version: "encryptionkey@0"
I20260927 10:51:07.406170 23859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:07.422266 23859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:07.425477 23859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:07.425669 23859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:07.426225 23859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:07.427037 23859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:07.427106 23859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.427178 23859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:07.427229 23859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.435756 23859 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:35401
I20260927 10:51:07.435782 23994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:35401 every 8 connection(s)
I20260927 10:51:07.436743 23859 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:51:07.439162 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23859
I20260927 10:51:07.439637 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:41033
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:39751
--webserver_port=36191
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:07.451432 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.438379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:54817 (local address 127.9.39.190:39751)
0927 10:51:07.438541 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.438548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.439760 (+ 1212us) server_negotiation.cc:407] Connection header received
0927 10:51:07.440694 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.440700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.440771 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.440875 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:07.442427 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.443378 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.444561 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.445041 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.445179 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.447600 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:07.447607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:07.447611 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:07.447643 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:07.449369 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.449834 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.449841 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.449845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.449954 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.450274 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.450279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.450283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.450539 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:07.450651 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.451139 (+ 488us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.451265 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:51:07.452176 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.438813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:54817)
0927 10:51:07.439567 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.439651 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.440486 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.441079 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.441091 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.441601 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:07.442277 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.442294 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.443496 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.443499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.444423 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.444432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.445188 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.445191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.445326 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.445937 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:07.445954 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:07.447426 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:07.449487 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.449491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.449503 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.449735 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.450057 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.450059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.450061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.450180 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.450651 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:07.450656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:07.450914 (+ 258us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.451064 (+ 150us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.451294 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":151,"threads_started":1}
I20260927 10:51:07.453326 23995 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39751
I20260927 10:51:07.453563 23995 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:07.454057 23995 heartbeater.cc:511] Master 127.9.39.190:39751 requested a full tablet report, sending...
I20260927 10:51:07.455021 23669 ts_manager.cc:194] Registered new tserver with Master: 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
I20260927 10:51:07.455569 23669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:54817
W20260927 10:51:07.550112 23998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:07.550354 23998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:07.550405 23998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:07.554363 23998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:07.554419 23998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:07.554476 23998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:51:07.559549 23998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:41033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=36191
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:07.560536 23998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:07.561535 23998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:07.570684 24006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.570798 23998 server_base.cc:1034] running on GCE node
W20260927 10:51:07.570672 24003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.570724 24004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.571312 23998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:07.571911 23998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:07.573098 23998 hybrid_clock.cc:648] HybridClock initialized: now 1790506267573085 us; error 24 us; skew 500 ppm
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:07.575740 23998 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:51:07.576536 23998 webserver.cc:460] Webserver started at http://127.9.39.131:36191/ using document root <none> and password file <none>
I20260927 10:51:07.577059 23998 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:07.577155 23998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:07.580772 23998 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260927 10:51:07.582681 24013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.583870 23998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:07.583987 23998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "7db46a991a64e99b022accb4ce3e99a5"
server_key_iv: "3b0766483d4fe69e4e7b790cce2716d7"
server_key_version: "encryptionkey@0"
I20260927 10:51:07.584494 23998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:07.596123 23998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:07.598861 23998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:07.599026 23998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:07.599574 23998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:07.600333 23998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:07.600378 23998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.600436 23998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:07.600478 23998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.609638 23998 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:41033
I20260927 10:51:07.609663 24126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:41033 every 8 connection(s)
I20260927 10:51:07.610652 23998 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:51:07.616184 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 23998
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:07.624449 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.612334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:40555 (local address 127.9.39.190:39751)
0927 10:51:07.612464 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.612467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.613379 (+ 912us) server_negotiation.cc:407] Connection header received
0927 10:51:07.614272 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.614276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.614335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.614438 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:07.615644 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.616144 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.617530 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.617784 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.617892 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.620623 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:07.620627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:07.620630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:07.620649 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:07.622218 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.622804 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.622808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.622811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.622921 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.623344 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.623350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.623354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.623576 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:07.623711 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.624146 (+ 435us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.624231 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 10:51:07.625571 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.612656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:40555)
0927 10:51:07.613244 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.613281 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.614072 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.614609 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.614616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.615036 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:07.615507 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.615520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.616283 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.616287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.617389 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.617400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.617925 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.617930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.618196 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.618824 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:07.618839 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:07.620430 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:07.622327 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.622331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.622342 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.622713 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.623018 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:07.623039 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:07.623043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:07.623220 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:07.623681 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:07.623686 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:07.623889 (+ 203us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.624089 (+ 200us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.624513 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":100,"threads_started":1}
I20260927 10:51:07.625696 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.616858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.9.39.190:39751)
0927 10:51:07.617174 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.617180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.617246 (+ 66us) server_negotiation.cc:407] Connection header received
0927 10:51:07.617397 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.617400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.617442 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.617546 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:07.618317 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.619029 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.620160 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.620435 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.620508 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.622186 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:07.622191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:07.622193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:07.622214 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:07.623894 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.624362 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.624366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.624367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.624462 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:07.624804 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:07.624808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:07.624809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:07.625031 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:07.625225 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.625341 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.625487 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":72,"threads_started":1}
I20260927 10:51:07.626822 24127 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39751
I20260927 10:51:07.627072 24127 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:07.627558 24127 heartbeater.cc:511] Master 127.9.39.190:39751 requested a full tablet report, sending...
I20260927 10:51:07.628479 23669 ts_manager.cc:194] Registered new tserver with Master: 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033)
I20260927 10:51:07.629000 23669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:40555
I20260927 10:51:07.638921 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:51:07.642697 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.639473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.9.39.190:39751)
0927 10:51:07.639590 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.639594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.639756 (+ 162us) server_negotiation.cc:407] Connection header received
0927 10:51:07.639877 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.639879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.639971 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.640054 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:51:07.640242 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.640710 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.641510 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.641717 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.641787 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.642034 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:51:07.642366 (+ 332us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:51:07.642450 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.642499 (+ 49us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.642544 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:51:07.643350 23669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:51:07.646389 23669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:51:07.654124 23782 tablet_service.cc:1467] Processing CreateTablet for tablet 90b618846216452abb29dbde00c9979f (DEFAULT_TABLE table=test-table [id=f7266a7341e94ac18db964b928755784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:07.654778 23782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b618846216452abb29dbde00c9979f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:07.657685 23749 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap starting.
I20260927 10:51:07.658078 23864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.652820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:41033 (local address 127.0.0.1:38438)
0927 10:51:07.652965 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.652979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.653049 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.653998 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.654003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.654022 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:07.654117 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.654122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.655582 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.655585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.656766 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.656776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.657674 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.657678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.657782 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.657831 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.657887 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.657959 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260927 10:51:07.658465 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.653178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35401 (local address 127.0.0.1:50056)
0927 10:51:07.653406 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.653418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.653497 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.654580 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.654582 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.654597 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:07.654660 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.654664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.655950 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.655953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.656900 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.656911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.657979 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.657982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.658071 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.658092 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.658251 (+ 159us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.658326 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260927 10:51:07.658668 23749 tablet_bootstrap.cc:654] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:07.658952 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.653591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.9.39.130:35401)
0927 10:51:07.653985 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.653996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.654017 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:51:07.654086 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.654092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.654283 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.654448 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:07.654807 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.655794 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.657025 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.657847 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.657979 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.658121 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.658627 (+ 506us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.658788 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":95,"threads_started":1}
I20260927 10:51:07.660073 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.652997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.9.39.131:41033)
0927 10:51:07.653380 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.653392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.653417 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:51:07.653507 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.653513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.653829 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.653996 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:07.654217 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.655401 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.656897 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.657504 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.658483 (+ 979us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.658815 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.658871 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.658970 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":145,"threads_started":1}
I20260927 10:51:07.660305 23749 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: No bootstrap required, opened a new log
I20260927 10:51:07.660400 23749 ts_tablet_manager.cc:1397] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:51:07.660815 23749 raft_consensus.cc:348] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.660966 23749 raft_consensus.cc:374] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:07.661015 23749 raft_consensus.cc:729] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0f1776bd34ade9498e5e5874b146b, State: Initialized, Role: FOLLOWER
I20260927 10:51:07.661170 23749 consensus_queue.cc:260] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.661509 23749 ts_tablet_manager.cc:1428] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:51:07.662099 23929 tablet_service.cc:1467] Processing CreateTablet for tablet 90b618846216452abb29dbde00c9979f (DEFAULT_TABLE table=test-table [id=f7266a7341e94ac18db964b928755784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:07.662714 24061 tablet_service.cc:1467] Processing CreateTablet for tablet 90b618846216452abb29dbde00c9979f (DEFAULT_TABLE table=test-table [id=f7266a7341e94ac18db964b928755784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:07.663249 23929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b618846216452abb29dbde00c9979f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:07.663666 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b618846216452abb29dbde00c9979f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:07.671144 24140 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Bootstrap starting.
I20260927 10:51:07.671144 24141 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Bootstrap starting.
I20260927 10:51:07.673856 24140 tablet_bootstrap.cc:654] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:07.674053 24141 tablet_bootstrap.cc:654] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:07.674815 24140 log.cc:824] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:07.674814 24141 log.cc:824] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:07.676514 24140 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: No bootstrap required, opened a new log
I20260927 10:51:07.676529 24141 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: No bootstrap required, opened a new log
I20260927 10:51:07.676714 24141 ts_tablet_manager.cc:1397] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:51:07.676806 24140 ts_tablet_manager.cc:1397] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:51:07.679819 24141 raft_consensus.cc:348] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.679801 24140 raft_consensus.cc:348] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.680047 24141 raft_consensus.cc:374] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:07.680042 24140 raft_consensus.cc:374] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:07.680107 24141 raft_consensus.cc:729] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fa368bc0c04bf8b9d5ff48eda4e0f2, State: Initialized, Role: FOLLOWER
I20260927 10:51:07.680109 24140 raft_consensus.cc:729] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3496156c65ef4f96ae0afe105be28699, State: Initialized, Role: FOLLOWER
I20260927 10:51:07.680528 24140 consensus_queue.cc:260] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.680531 24141 consensus_queue.cc:260] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.681140 23995 heartbeater.cc:503] Master 127.9.39.190:39751 was elected leader, sending a full tablet report...
I20260927 10:51:07.681371 24127 heartbeater.cc:503] Master 127.9.39.190:39751 was elected leader, sending a full tablet report...
I20260927 10:51:07.681422 24140 ts_tablet_manager.cc:1428] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:51:07.681448 24141 ts_tablet_manager.cc:1428] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260927 10:51:07.687561 23996 tablet.cc:2369] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:07.740693 23863 raft_consensus.cc:482] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:07.740886 23863 raft_consensus.cc:504] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.741914 23863 leader_election.cc:288] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033), 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
I20260927 10:51:07.745983 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.742281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:59697 (local address 127.9.39.130:35401)
0927 10:51:07.742419 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.742423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.742447 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:51:07.742527 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.742529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.742571 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.742640 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:07.742967 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.743661 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.744895 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.745395 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.745489 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.745526 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.745784 (+ 258us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.745857 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 10:51:07.745988 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.742139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35401 (local address 127.9.39.129:59697)
0927 10:51:07.742249 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.742267 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.742389 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.742682 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.742685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.742724 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:07.742843 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.742849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.743766 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.743770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.744770 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.744780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.745505 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.745509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.745611 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.745662 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.745749 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.745833 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":19}
I20260927 10:51:07.746279 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.742139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:41033 (local address 127.9.39.129:47357)
0927 10:51:07.742859 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.742871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:07.743002 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:07.743239 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:07.743242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:07.743259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:07.743383 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.743389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.744212 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.744214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.745088 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:07.745099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.745848 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.745852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:07.745963 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.745984 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 10:51:07.746064 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 10:51:07.746138 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":377,"thread_start_us":110,"threads_started":1}
I20260927 10:51:07.746429 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.742281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:47357 (local address 127.9.39.131:41033)
0927 10:51:07.742431 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.742435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.742984 (+ 549us) server_negotiation.cc:407] Connection header received
0927 10:51:07.743098 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.743100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.743141 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.743200 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:07.743486 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.744101 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.745213 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.745721 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.745830 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.745874 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.746229 (+ 355us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.746304 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 10:51:07.746726 23949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b618846216452abb29dbde00c9979f" candidate_uuid: "49a0f1776bd34ade9498e5e5874b146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3496156c65ef4f96ae0afe105be28699" is_pre_election: true
I20260927 10:51:07.746966 24081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b618846216452abb29dbde00c9979f" candidate_uuid: "49a0f1776bd34ade9498e5e5874b146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" is_pre_election: true
I20260927 10:51:07.747042 23949 raft_consensus.cc:2454] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a0f1776bd34ade9498e5e5874b146b in term 0.
I20260927 10:51:07.747253 24081 raft_consensus.cc:2454] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a0f1776bd34ade9498e5e5874b146b in term 0.
I20260927 10:51:07.747563 23745 leader_election.cc:302] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3496156c65ef4f96ae0afe105be28699, 49a0f1776bd34ade9498e5e5874b146b; no voters:
I20260927 10:51:07.747869 23863 raft_consensus.cc:2781] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:51:07.747954 23863 raft_consensus.cc:482] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:51:07.747999 23863 raft_consensus.cc:3037] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:07.748997 23863 raft_consensus.cc:504] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.749378 23863 leader_election.cc:288] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 1 election: Requested vote from peers 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033), 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
I20260927 10:51:07.749661 24081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b618846216452abb29dbde00c9979f" candidate_uuid: "49a0f1776bd34ade9498e5e5874b146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
I20260927 10:51:07.749760 23949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b618846216452abb29dbde00c9979f" candidate_uuid: "49a0f1776bd34ade9498e5e5874b146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3496156c65ef4f96ae0afe105be28699"
I20260927 10:51:07.749794 24081 raft_consensus.cc:3037] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:07.749903 23949 raft_consensus.cc:3037] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:07.750921 24081 raft_consensus.cc:2454] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a0f1776bd34ade9498e5e5874b146b in term 1.
I20260927 10:51:07.751004 23949 raft_consensus.cc:2454] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a0f1776bd34ade9498e5e5874b146b in term 1.
I20260927 10:51:07.751199 23743 leader_election.cc:302] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 49a0f1776bd34ade9498e5e5874b146b; no voters:
I20260927 10:51:07.751379 23863 raft_consensus.cc:2781] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Leader election won for term 1
W20260927 10:51:07.751535 23858 tablet.cc:2369] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:07.751602 23863 raft_consensus.cc:686] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 LEADER]: Becoming Leader. State: Replica: 49a0f1776bd34ade9498e5e5874b146b, State: Running, Role: LEADER
I20260927 10:51:07.751853 23863 consensus_queue.cc:237] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:07.754842 23669 catalog_manager.cc:5495] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b reported cstate change: term changed from 0 to 1, leader changed from <none> to 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129). New cstate: current_term: 1 leader_uuid: "49a0f1776bd34ade9498e5e5874b146b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } health_report { overall_health: HEALTHY } } }
I20260927 10:51:07.788854 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.785272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.9.39.129:46581)
0927 10:51:07.785402 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:07.785406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:07.785421 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:51:07.785567 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:07.785569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:07.785623 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:07.785734 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:51:07.785974 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.786492 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.787423 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:07.787651 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:07.787753 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:07.788014 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:51:07.788484 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:51:07.788606 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:07.788663 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 10:51:07.788716 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:51:07.792514 24081 raft_consensus.cc:1261] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Refusing update from remote peer 49a0f1776bd34ade9498e5e5874b146b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:07.792515 23949 raft_consensus.cc:1261] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Refusing update from remote peer 49a0f1776bd34ade9498e5e5874b146b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:07.793165 23863 consensus_queue.cc:1035] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [LEADER]: Connected to new peer: Peer: permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:07.793385 24147 consensus_queue.cc:1035] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:07.800693 24150 mvcc.cc:205] Tried to move back new op lower bound from 7333913672872169472 to 7333913672714158080. Current Snapshot: MvccSnapshot[applied={T|T < 7333913672872169472}]
I20260927 10:51:07.803337 24151 mvcc.cc:205] Tried to move back new op lower bound from 7333913672872169472 to 7333913672714158080. Current Snapshot: MvccSnapshot[applied={T|T < 7333913672872169472}]
I20260927 10:51:07.808310 23669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:07.808466 23669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:07.810783 23669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Sending DeleteTablet for 3 replicas of tablet 90b618846216452abb29dbde00c9979f
I20260927 10:51:07.811424 24061 tablet_service.cc:1514] Processing DeleteTablet for tablet 90b618846216452abb29dbde00c9979f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:07 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:38438
I20260927 10:51:07.811472 23782 tablet_service.cc:1514] Processing DeleteTablet for tablet 90b618846216452abb29dbde00c9979f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:07 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:44788
I20260927 10:51:07.811537 23929 tablet_service.cc:1514] Processing DeleteTablet for tablet 90b618846216452abb29dbde00c9979f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:07 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:50056
I20260927 10:51:07.811906 24161 tablet_replica.cc:330] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: stopping tablet replica
I20260927 10:51:07.812089 24160 tablet_replica.cc:330] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: stopping tablet replica
I20260927 10:51:07.812167 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23707
I20260927 10:51:07.812240 24161 raft_consensus.cc:2229] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:07.812500 24161 raft_consensus.cc:2258] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:07.813426 24162 tablet_replica.cc:330] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: stopping tablet replica
I20260927 10:51:07.813691 24162 raft_consensus.cc:2229] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:07.813824 24161 ts_tablet_manager.cc:1905] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:07.813968 24162 raft_consensus.cc:2258] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:07.815270 24162 ts_tablet_manager.cc:1905] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:07.816929 24161 ts_tablet_manager.cc:1918] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:07.817054 24161 log.cc:1196] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/90b618846216452abb29dbde00c9979f
I20260927 10:51:07.817323 24162 ts_tablet_manager.cc:1918] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:07.817415 24161 ts_tablet_manager.cc:1939] T 90b618846216452abb29dbde00c9979f P 31fa368bc0c04bf8b9d5ff48eda4e0f2: Deleting consensus metadata
I20260927 10:51:07.817436 24162 log.cc:1196] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/90b618846216452abb29dbde00c9979f
I20260927 10:51:07.817711 24162 ts_tablet_manager.cc:1939] T 90b618846216452abb29dbde00c9979f P 3496156c65ef4f96ae0afe105be28699: Deleting consensus metadata
I20260927 10:51:07.818648 23653 catalog_manager.cc:4841] TS 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033): tablet 90b618846216452abb29dbde00c9979f (table test-table [id=f7266a7341e94ac18db964b928755784]) successfully deleted
I20260927 10:51:07.818766 23655 catalog_manager.cc:4841] TS 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401): tablet 90b618846216452abb29dbde00c9979f (table test-table [id=f7266a7341e94ac18db964b928755784]) successfully deleted
W20260927 10:51:07.820533 23656 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:46581 (error 108)
I20260927 10:51:07.820590 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23859
I20260927 10:51:07.821883 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:07.821647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46581 (local address 127.0.0.1:44804)
0927 10:51:07.821763 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:07.821819 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:46581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 10:51:07.822047 23656 catalog_manager.cc:4584] TS 49a0f1776bd34ade9498e5e5874b146b (127.9.39.129:46581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90b618846216452abb29dbde00c9979f: Network error: Client connection negotiation failed: client connection to 127.9.39.129:46581: connect: Connection refused (error 111)
I20260927 10:51:07.826781 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23998
I20260927 10:51:07.832057 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 23636
I20260927 10:51:07.838022 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39751
--webserver_interface=127.9.39.190
--webserver_port=41507
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:07.943414 24164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:07.943647 24164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:07.943684 24164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:07.947445 24164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:07.947490 24164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:07.947510 24164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:07.947525 24164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:07.947538 24164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:07.952504 24164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=41507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:07.953373 24164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:07.954329 24164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:07.962312 24172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:07.962278 24169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.962440 24164 server_base.cc:1034] running on GCE node
W20260927 10:51:07.962363 24170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:07.963275 24164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:07.964372 24164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:07.965600 24164 hybrid_clock.cc:648] HybridClock initialized: now 1790506267965566 us; error 48 us; skew 500 ppm
Sep 27 10:51:07 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:07.968503 24164 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:07.969370 24164 webserver.cc:460] Webserver started at http://127.9.39.190:41507/ using document root <none> and password file <none>
I20260927 10:51:07.969902 24164 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:07.969990 24164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:07.973714 24164 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:07.975767 24179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:07.976907 24164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:51:07.977035 24164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "7951fbdb097c49fc8eb550535897c5b1"
format_stamp: "Formatted at 2026-09-27 10:51:05 on dist-test-slave-p9q6"
server_key: "7bd6e68b8567e120ae87b1a0c6b9eb40"
server_key_iv: "b16db7f2a7a9cede26ff7cfcb77c5afd"
server_key_version: "encryptionkey@0"
I20260927 10:51:07.977556 24164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:07.992266 24164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:07.995253 24164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:07.995460 24164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:08.002287 24164 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39751
I20260927 10:51:08.002310 24231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39751 every 8 connection(s)
I20260927 10:51:08.003312 24164 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:08.004494 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24164
I20260927 10:51:08.005013 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:46581
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:39751
--webserver_port=46703
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:08.008318 24232 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:51:08.009969 24232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap starting.
I20260927 10:51:08.027393 24232 log.cc:824] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:08.038349 24232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:51:08.038801 24232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1: Bootstrap complete.
I20260927 10:51:08.043049 24232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:08.043476 24232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7951fbdb097c49fc8eb550535897c5b1, State: Initialized, Role: FOLLOWER
I20260927 10:51:08.043996 24232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:08.044142 24232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:51:08.044236 24232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:51:08.044360 24232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 2 FOLLOWER]: Advancing to term 3
I20260927 10:51:08.046226 24232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:08.046562 24232 leader_election.cc:302] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7951fbdb097c49fc8eb550535897c5b1; no voters:
I20260927 10:51:08.046950 24232 leader_election.cc:288] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 10:51:08.047032 24236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 10:51:08.047212 24236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [term 3 LEADER]: Becoming Leader. State: Replica: 7951fbdb097c49fc8eb550535897c5b1, State: Running, Role: LEADER
I20260927 10:51:08.047489 24236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } }
I20260927 10:51:08.048149 24232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:08.048396 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7951fbdb097c49fc8eb550535897c5b1. Latest consensus state: current_term: 3 leader_uuid: "7951fbdb097c49fc8eb550535897c5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } } }
I20260927 10:51:08.048512 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:08.048868 24240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:08.048892 24237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7951fbdb097c49fc8eb550535897c5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7951fbdb097c49fc8eb550535897c5b1" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39751 } } }
I20260927 10:51:08.049103 24237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951fbdb097c49fc8eb550535897c5b1 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:08.053969 24240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:08.054823 24240 catalog_manager.cc:1259] Loaded cluster ID: 4fba3c3ce577486db9445971c056f4aa
I20260927 10:51:08.054886 24240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 10:51:08.055819 24240 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5a1fb829ad google::LogMessage::Fail() at ??:0
@ 0x7f5a1fb8588a google::LogMessage::SendToLog() at ??:0
@ 0x7f5a1fb824de google::LogMessage::Flush() at ??:0
@ 0x7f5a1fb83a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5a2a4c1fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5a2a4c2d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5a2a4bf579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5a2a503640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5a21dd43d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5a21dd4c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5a21dd63f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5a21dc5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5a234096db start_thread at ??:0
@ 0x7f5a1e68061f clone at ??:0
W20260927 10:51:08.141404 24234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:08.141651 24234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:08.141696 24234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:08.145771 24234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:08.145829 24234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:08.145892 24234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:51:08.151149 24234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:46581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=46703
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:08.152194 24234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:08.153173 24234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:08.163803 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:08.163810 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:08.163964 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:08.165248 24234 server_base.cc:1034] running on GCE node
I20260927 10:51:08.165663 24234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:08.166342 24234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:08.167519 24234 hybrid_clock.cc:648] HybridClock initialized: now 1790506268167492 us; error 50 us; skew 500 ppm
Sep 27 10:51:08 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:08.170307 24234 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:51:08.171324 24234 webserver.cc:460] Webserver started at http://127.9.39.129:46703/ using document root <none> and password file <none>
I20260927 10:51:08.171892 24234 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:08.171985 24234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:08.176074 24234 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260927 10:51:08.178471 24283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.179797 24234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 10:51:08.179957 24234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "49a0f1776bd34ade9498e5e5874b146b"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "ce5ed4da0c6fed0f5de2772a12d444da"
server_key_iv: "1f868ab71ac099a0a8c6ad677109f8ae"
server_key_version: "encryptionkey@0"
I20260927 10:51:08.180511 24234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:08.210156 24234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:08.213500 24234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:08.213711 24234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:08.214309 24234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:08.215643 24290 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 10:51:08.219776 24234 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 10:51:08.219836 24234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260927 10:51:08.219904 24234 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 10:51:08.221520 24234 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 10:51:08.221575 24234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260927 10:51:08.221904 24290 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap starting.
I20260927 10:51:08.232314 24234 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:46581
I20260927 10:51:08.232569 24397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:46581 every 8 connection(s)
I20260927 10:51:08.233608 24234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:51:08.242277 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24234
I20260927 10:51:08.242239 24290 log.cc:824] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:08.242897 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:35401
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:39751
--webserver_port=36973
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:08.249831 24290 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:51:08.250269 24290 tablet_bootstrap.cc:492] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Bootstrap complete.
I20260927 10:51:08.251739 24290 ts_tablet_manager.cc:1397] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Time spent bootstrapping tablet: real 0.030s user 0.021s sys 0.007s
I20260927 10:51:08.255147 24290 raft_consensus.cc:348] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:08.255709 24290 raft_consensus.cc:729] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0f1776bd34ade9498e5e5874b146b, State: Initialized, Role: FOLLOWER
I20260927 10:51:08.256249 24290 consensus_queue.cc:260] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:08.257351 24290 ts_tablet_manager.cc:1428] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
W20260927 10:51:08.277154 24400 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:51:08.236031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:47921)
0927 10:51:08.236607 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.236665 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:08.237685 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:08.276171 (+ 38486us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":149,"threads_started":1}
W20260927 10:51:08.277828 24398 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:39751 (0 consecutive failures): Network error: Failed to ping master at 127.9.39.190:39751: Client connection negotiation failed: client connection to 127.9.39.190:39751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 10:51:08.278548 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.278279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:47821)
0927 10:51:08.278413 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.278485 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 10:51:08.279232 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.279043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:60409)
0927 10:51:08.279143 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.279186 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 10:51:08.279488 24398 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:51:08.358336 24402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:08.358616 24402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:08.358690 24402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:08.362800 24402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:08.362850 24402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:08.362943 24402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:51:08.368184 24402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:35401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=36973
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:08.369171 24402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:08.370159 24402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:08.379750 24411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:08.379839 24408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:08.379917 24409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:08.380174 24402 server_base.cc:1034] running on GCE node
I20260927 10:51:08.380529 24402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:08.381132 24402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:08.382282 24402 hybrid_clock.cc:648] HybridClock initialized: now 1790506268382239 us; error 55 us; skew 500 ppm
Sep 27 10:51:08 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:08.385005 24402 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:51:08.385826 24402 webserver.cc:460] Webserver started at http://127.9.39.130:36973/ using document root <none> and password file <none>
I20260927 10:51:08.386322 24402 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:08.386379 24402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:08.389999 24402 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:08.391987 24418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.393153 24402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:51:08.393255 24402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3496156c65ef4f96ae0afe105be28699"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "b56841a198f76a0e5d5bcf7bb49af3e0"
server_key_iv: "efec9393ebef3d1ad22b68f302e14e45"
server_key_version: "encryptionkey@0"
I20260927 10:51:08.393770 24402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:08.404670 24402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:08.407294 24402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:08.407466 24402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:08.407950 24402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:08.408659 24402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:08.408703 24402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.408739 24402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:08.408761 24402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.417799 24402 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:35401
I20260927 10:51:08.417826 24531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:35401 every 8 connection(s)
I20260927 10:51:08.418814 24402 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:51:08.422179 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24402
I20260927 10:51:08.422644 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.420914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:35403)
0927 10:51:08.421603 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.421704 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":160,"threads_started":1}
I20260927 10:51:08.422670 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:41033
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:39751
--webserver_port=36191
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:08.423085 24532 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:39751 (0 consecutive failures): Network error: Failed to ping master at 127.9.39.190:39751: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
I20260927 10:51:08.423750 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.423469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:57971)
0927 10:51:08.423603 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.423669 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 10:51:08.424417 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.424223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:51555)
0927 10:51:08.424324 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.424365 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 10:51:08.424590 24532 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:51:08.533079 24535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:08.533354 24535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:08.533396 24535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:08.537483 24535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:08.537544 24535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:08.537600 24535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:51:08.542881 24535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:41033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=36191
--tserver_master_addrs=127.9.39.190:39751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:08.543990 24535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:08.545066 24535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:08.554826 24540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:08.554826 24541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:08.555102 24543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:08.555096 24535 server_base.cc:1034] running on GCE node
I20260927 10:51:08.555687 24535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:08.556313 24535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:08.557518 24535 hybrid_clock.cc:648] HybridClock initialized: now 1790506268557483 us; error 53 us; skew 500 ppm
Sep 27 10:51:08 dist-test-slave-p9q6 krb5kdc[23089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:08.560461 24535 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:51:08.561252 24535 webserver.cc:460] Webserver started at http://127.9.39.131:36191/ using document root <none> and password file <none>
I20260927 10:51:08.561748 24535 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:08.561813 24535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:08.565538 24535 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:08.567914 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.569294 24535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:51:08.569449 24535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2"
format_stamp: "Formatted at 2026-09-27 10:51:06 on dist-test-slave-p9q6"
server_key: "7db46a991a64e99b022accb4ce3e99a5"
server_key_iv: "3b0766483d4fe69e4e7b790cce2716d7"
server_key_version: "encryptionkey@0"
I20260927 10:51:08.569981 24535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:08.585816 24535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:08.588864 24535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:08.589090 24535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:08.589717 24535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:08.590559 24535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:08.590605 24535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.590675 24535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:08.590731 24535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:08.600919 24535 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:41033
I20260927 10:51:08.600941 24663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:41033 every 8 connection(s)
I20260927 10:51:08.601949 24535 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:51:08.605540 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.604070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:45309)
0927 10:51:08.604718 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.604825 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":105,"threads_started":1}
W20260927 10:51:08.606072 24664 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:39751 (0 consecutive failures): Network error: Failed to ping master at 127.9.39.190:39751: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
I20260927 10:51:08.606630 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.606436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:38101)
0927 10:51:08.606535 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.606579 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 10:51:08.607311 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:08.607084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:54025)
0927 10:51:08.607204 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:08.607253 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260927 10:51:08.607491 24664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 10:51:08.609571 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24535
I20260927 10:51:09.280536 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.280101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:53889)
0927 10:51:09.280363 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:09.280428 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260927 10:51:09.425518 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.425102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:44399)
0927 10:51:09.425364 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:09.425421 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260927 10:51:09.608362 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.607985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:50877)
0927 10:51:09.608210 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:09.608262 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":66,"threads_started":1}
I20260927 10:51:09.910331 24671 raft_consensus.cc:482] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:09.910590 24671 raft_consensus.cc:504] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:09.911725 24671 leader_election.cc:288] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033), 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
Sep 27 10:51:09 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.130@KRBTEST.COM
Sep 27 10:51:09 dist-test-slave-p9q6 krb5kdc[23089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.131@KRBTEST.COM
I20260927 10:51:09.926208 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.911888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:41033 (local address 127.9.39.129:38969)
0927 10:51:09.912171 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:09.912186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:09.912292 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:09.913859 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:09.913865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:09.914402 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:09.914889 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:09.914901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.916163 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.916166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:09.917292 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:09.917301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.917992 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.917996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:09.918207 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:09.918978 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:09.918999 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:09.921178 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:09.923321 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:09.923324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:09.923326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:09.923784 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:09.924156 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:09.924159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:09.924161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:09.924367 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:09.925115 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:09.925122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:09.925467 (+ 345us) client_negotiation.cc:769] Sending connection context
0927 10:51:09.925677 (+ 210us) client_negotiation.cc:240] Negotiation successful
0927 10:51:09.926056 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":232,"spinlock_wait_cycles":66304,"thread_start_us":79,"threads_started":1}
I20260927 10:51:09.926288 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.912042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:38969 (local address 127.9.39.131:41033)
0927 10:51:09.912432 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:09.912457 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:09.912502 (+ 45us) server_negotiation.cc:407] Connection header received
0927 10:51:09.913186 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:09.913191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:09.913521 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:09.913816 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:09.915051 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.916012 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.917422 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.917870 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.918048 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:09.921356 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:09.921362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:09.921378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:09.921411 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:09.923229 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:09.923901 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:09.923906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:09.923912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:09.924060 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:09.924466 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:09.924469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:09.924471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:09.924972 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:09.925128 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:09.925821 (+ 693us) server_negotiation.cc:299] Negotiation successful
0927 10:51:09.926094 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":71,"threads_started":1}
I20260927 10:51:09.926321 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.912184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:49543 (local address 127.9.39.130:35401)
0927 10:51:09.912533 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:09.912552 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:09.912582 (+ 30us) server_negotiation.cc:407] Connection header received
0927 10:51:09.913186 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:09.913191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:09.913520 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:09.913815 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:09.915054 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.916000 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.917390 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.917868 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.918047 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:09.921101 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:09.921107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:09.921126 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:09.921152 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:09.923136 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:09.923901 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:09.923906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:09.923912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:09.924060 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:09.924466 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:09.924469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:09.924471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:09.924972 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:09.925128 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:09.925821 (+ 693us) server_negotiation.cc:299] Negotiation successful
0927 10:51:09.926094 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":101,"threads_started":1}
I20260927 10:51:09.926208 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:09.912001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:35401 (local address 127.9.39.129:49543)
0927 10:51:09.912314 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:09.912327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:09.912421 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:09.913859 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:09.913865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:09.914402 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:09.914885 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:09.914900 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.916141 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.916146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:09.917282 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:09.917291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:09.917991 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:09.917995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:09.918221 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:09.918978 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:09.918997 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:09.920827 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:09.923262 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:09.923267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:09.923296 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:09.923784 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:09.924156 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:09.924158 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:09.924161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:09.924367 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:09.925115 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:09.925122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:09.925467 (+ 345us) client_negotiation.cc:769] Sending connection context
0927 10:51:09.925677 (+ 210us) client_negotiation.cc:240] Negotiation successful
0927 10:51:09.926004 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":225,"spinlock_wait_cycles":30208,"thread_start_us":67,"threads_started":1}
W20260927 10:51:09.929481 24284 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
W20260927 10:51:09.929649 24286 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
I20260927 10:51:09.929746 24286 leader_election.cc:302] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49a0f1776bd34ade9498e5e5874b146b; no voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 3496156c65ef4f96ae0afe105be28699
I20260927 10:51:09.930042 24671 raft_consensus.cc:2726] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:51:10.281633 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:10.281357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:35109)
0927 10:51:10.281490 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:10.281549 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 10:51:10.426671 24676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:10.426278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:50995)
0927 10:51:10.426520 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:10.426574 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
I20260927 10:51:10.609545 24677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:10.609137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:59925)
0927 10:51:10.609383 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:10.609458 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":68,"threads_started":1}
I20260927 10:51:11.283263 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:11.282508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:48453)
0927 10:51:11.283041 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:11.283122 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":79,"threads_started":1}
I20260927 10:51:11.427930 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:11.427520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:56239)
0927 10:51:11.427773 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:11.427830 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260927 10:51:11.610778 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:11.610326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:34027)
0927 10:51:11.610599 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:11.610651 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260927 10:51:12.138847 24681 raft_consensus.cc:482] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:12.138972 24681 raft_consensus.cc:504] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:12.139348 24681 leader_election.cc:288] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033), 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
W20260927 10:51:12.140229 24284 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
W20260927 10:51:12.140322 24286 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
I20260927 10:51:12.140434 24286 leader_election.cc:302] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49a0f1776bd34ade9498e5e5874b146b; no voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 3496156c65ef4f96ae0afe105be28699
I20260927 10:51:12.140626 24681 raft_consensus.cc:2726] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:51:12.284406 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:12.283959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:38041)
0927 10:51:12.284237 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:12.284313 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":131,"threads_started":1}
I20260927 10:51:12.429078 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:12.428657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:39821)
0927 10:51:12.428932 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:12.428989 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260927 10:51:12.612020 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:12.611527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:37241)
0927 10:51:12.611852 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:12.611917 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260927 10:51:13.285740 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:13.285274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:57377)
0927 10:51:13.285576 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:13.285633 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":76,"threads_started":1}
I20260927 10:51:13.430258 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:13.429821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:41423)
0927 10:51:13.430099 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:13.430158 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260927 10:51:13.613238 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:13.612824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:49491)
0927 10:51:13.613063 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:13.613115 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260927 10:51:14.286887 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:14.286440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:52559)
0927 10:51:14.286669 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:14.286791 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":70,"threads_started":1}
I20260927 10:51:14.431311 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:14.430945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:57931)
0927 10:51:14.431159 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:14.431208 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
I20260927 10:51:14.614377 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:14.613907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:59179)
0927 10:51:14.614241 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:14.614293 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":67,"threads_started":1}
I20260927 10:51:14.919749 24691 raft_consensus.cc:482] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:14.919920 24691 raft_consensus.cc:504] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:14.920390 24691 leader_election.cc:288] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033), 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
W20260927 10:51:14.921201 24286 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
W20260927 10:51:14.921290 24284 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
I20260927 10:51:14.921353 24284 leader_election.cc:302] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49a0f1776bd34ade9498e5e5874b146b; no voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 3496156c65ef4f96ae0afe105be28699
I20260927 10:51:14.921541 24691 raft_consensus.cc:2726] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:51:15.288000 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:15.287602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:50289)
0927 10:51:15.287857 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:15.287909 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":102,"threads_started":1}
I20260927 10:51:15.432581 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:15.432109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:35051)
0927 10:51:15.432441 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:15.432496 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":90,"threads_started":1}
I20260927 10:51:15.615525 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:15.615096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:42451)
0927 10:51:15.615383 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:15.615437 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":69,"threads_started":1}
I20260927 10:51:16.289260 24695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:16.288780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:60433)
0927 10:51:16.289038 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:16.289091 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260927 10:51:16.433753 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:16.433363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:36393)
0927 10:51:16.433598 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:16.433659 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260927 10:51:16.616631 24697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:16.616247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:52815)
0927 10:51:16.616481 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:16.616537 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260927 10:51:17.290429 24698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:17.290007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:37875)
0927 10:51:17.290283 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:17.290337 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260927 10:51:17.434911 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:17.434468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:56581)
0927 10:51:17.434747 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:17.434823 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260927 10:51:17.617913 24700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:17.617512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:35057)
0927 10:51:17.617754 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:17.617825 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260927 10:51:18.291747 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:18.291369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:35181)
0927 10:51:18.291596 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:18.291646 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260927 10:51:18.436232 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:18.435788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:37247)
0927 10:51:18.436046 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:18.436116 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260927 10:51:18.619089 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:18.618660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:45339)
0927 10:51:18.618948 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:18.619001 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":132,"threads_started":1}
I20260927 10:51:19.120842 24704 raft_consensus.cc:482] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:19.120976 24704 raft_consensus.cc:504] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fa368bc0c04bf8b9d5ff48eda4e0f2" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 41033 } } peers { permanent_uuid: "3496156c65ef4f96ae0afe105be28699" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 35401 } } peers { permanent_uuid: "49a0f1776bd34ade9498e5e5874b146b" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46581 } }
I20260927 10:51:19.121440 24704 leader_election.cc:288] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033), 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401)
W20260927 10:51:19.122208 24284 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31fa368bc0c04bf8b9d5ff48eda4e0f2 (127.9.39.131:41033): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
W20260927 10:51:19.122337 24286 leader_election.cc:341] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3496156c65ef4f96ae0afe105be28699 (127.9.39.130:35401): Not found: Tablet not found: 90b618846216452abb29dbde00c9979f
I20260927 10:51:19.122429 24286 leader_election.cc:302] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49a0f1776bd34ade9498e5e5874b146b; no voters: 31fa368bc0c04bf8b9d5ff48eda4e0f2, 3496156c65ef4f96ae0afe105be28699
I20260927 10:51:19.122591 24704 raft_consensus.cc:2726] T 90b618846216452abb29dbde00c9979f P 49a0f1776bd34ade9498e5e5874b146b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:51:19.292997 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:19.292523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:41889)
0927 10:51:19.292814 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:19.292882 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":87,"threads_started":1}
I20260927 10:51:19.437382 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:19.436973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:48509)
0927 10:51:19.437232 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:19.437281 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
I20260927 10:51:19.620302 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:19.619836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:34957)
0927 10:51:19.620135 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:19.620204 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":113,"threads_started":1}
I20260927 10:51:20.294054 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:20.293697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:54867)
0927 10:51:20.293917 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:20.293964 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260927 10:51:20.438599 24709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:20.438187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:33017)
0927 10:51:20.438449 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:20.438505 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260927 10:51:20.621448 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:20.621027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:43521)
0927 10:51:20.621268 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:20.621350 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
I20260927 10:51:21.295280 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:21.294882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:36593)
0927 10:51:21.295133 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:21.295191 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260927 10:51:21.439845 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:21.439426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:48101)
0927 10:51:21.439686 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:21.439737 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260927 10:51:21.622539 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:21.622159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:50995)
0927 10:51:21.622392 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:21.622448 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260927 10:51:22.296393 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:22.295980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:47739)
0927 10:51:22.296252 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:22.296304 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":68,"threads_started":1}
I20260927 10:51:22.440918 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:22.440528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:43249)
0927 10:51:22.440765 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:22.440828 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260927 10:51:22.623780 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:22.623313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:58967)
0927 10:51:22.623589 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:22.623671 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260927 10:51:23.297988 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:23.297478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.129:36811)
0927 10:51:23.297797 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:23.297859 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":125,"threads_started":1}
I20260927 10:51:23.442308 24718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:23.441879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.130:60825)
0927 10:51:23.442152 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:23.442215 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260927 10:51:23.622658 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 24234
I20260927 10:51:23.625075 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:23.624656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39751 (local address 127.9.39.131:33529)
0927 10:51:23.624934 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:23.624989 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
I20260927 10:51:23.628516 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 24402
I20260927 10:51:23.634094 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 24535
2026-09-27T10:51:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19550 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:51:24 dist-test-slave-p9q6 krb5kdc[24722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:51:24 dist-test-slave-p9q6 krb5kdc[24722](info): set up 2 sockets
Sep 27 10:51:24 dist-test-slave-p9q6 krb5kdc[24722](info): commencing operation
krb5kdc: starting...
W20260927 10:51:25.079206 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:51:25.201588 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45913
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:25.312088 24735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:25.312407 24735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:25.312479 24735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:25.316509 24735 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 10:51:25.316601 24735 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 10:51:25.316627 24735 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 10:51:25.316672 24735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:25.316713 24735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:25.316754 24735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:25.316787 24735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:25.316821 24735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:25.322080 24735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:25.323175 24735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:25.324218 24735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:25.332965 24741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:25.333060 24743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:25.333200 24735 server_base.cc:1034] running on GCE node
W20260927 10:51:25.332965 24740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:25.333895 24735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:25.335048 24735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:25.336239 24735 hybrid_clock.cc:648] HybridClock initialized: now 1790506285336226 us; error 24 us; skew 500 ppm
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:25.338946 24735 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:25.339890 24735 webserver.cc:460] Webserver started at http://127.9.39.190:34859/ using document root <none> and password file <none>
I20260927 10:51:25.340471 24735 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:25.340561 24735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:25.340797 24735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:25.342921 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "bd29ee5fcb234487a5ff59c716bb5036"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "713e73e24e015a86677be964978ae4bf"
server_key_iv: "9dd3f5a1f90d613d515f9f9ecf217cfb"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.343513 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "bd29ee5fcb234487a5ff59c716bb5036"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "713e73e24e015a86677be964978ae4bf"
server_key_iv: "9dd3f5a1f90d613d515f9f9ecf217cfb"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.347270 24735 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:51:25.349586 24750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.350857 24735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:25.350987 24735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "bd29ee5fcb234487a5ff59c716bb5036"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "713e73e24e015a86677be964978ae4bf"
server_key_iv: "9dd3f5a1f90d613d515f9f9ecf217cfb"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.351097 24735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:25.376715 24735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:25.379698 24735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:25.379917 24735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:25.386960 24735 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:45913
I20260927 10:51:25.387302 24802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:45913 every 8 connection(s)
I20260927 10:51:25.387938 24735 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:25.390735 24803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:25.396088 24803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap starting.
I20260927 10:51:25.397924 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24735
I20260927 10:51:25.398047 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:51:25.398348 9374 external_mini_cluster.cc:1428] Setting key 5b1459c8642b70ac4d51c34ebda0ce95
I20260927 10:51:25.398831 24803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:25.399695 24803 log.cc:824] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:25.401511 24803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: No bootstrap required, opened a new log
I20260927 10:51:25.404989 24803 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:25.405220 24803 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:25.405279 24803 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd29ee5fcb234487a5ff59c716bb5036, State: Initialized, Role: FOLLOWER
I20260927 10:51:25.405757 24803 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:25.405903 24803 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:51:25.405988 24803 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:51:25.406101 24803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:25.407191 24803 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:25.407542 24803 leader_election.cc:302] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd29ee5fcb234487a5ff59c716bb5036; no voters:
I20260927 10:51:25.407894 24803 leader_election.cc:288] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:51:25.407994 24808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:25.408253 24808 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 LEADER]: Becoming Leader. State: Replica: bd29ee5fcb234487a5ff59c716bb5036, State: Running, Role: LEADER
I20260927 10:51:25.408588 24808 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:25.409251 24803 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:25.410246 24810 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd29ee5fcb234487a5ff59c716bb5036. Latest consensus state: current_term: 1 leader_uuid: "bd29ee5fcb234487a5ff59c716bb5036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } } }
I20260927 10:51:25.410372 24810 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:25.410768 24809 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd29ee5fcb234487a5ff59c716bb5036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } } }
I20260927 10:51:25.410864 24809 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:25.411562 24817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:25.413230 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:25.399490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.9.39.190:45913)
0927 10:51:25.399978 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:25.399993 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:25.400049 (+ 56us) server_negotiation.cc:407] Connection header received
0927 10:51:25.400837 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:25.400857 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:25.401195 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:25.401544 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:25.402392 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.403439 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.404282 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.404743 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.404927 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:25.407861 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:25.407867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:25.407883 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:25.407915 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:25.410066 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:25.410639 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:25.410645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:25.410651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:25.410817 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:25.411189 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:25.411191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:25.411193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:25.411702 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:25.412081 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:25.412250 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 10:51:25.412550 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":142,"threads_started":1}
I20260927 10:51:25.414565 24817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:25.420245 24817 catalog_manager.cc:1347] Generated new cluster ID: dfae65ba2ff44766b312fd9ffb58bc2a
I20260927 10:51:25.420311 24817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:51:25.437688 24817 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:51:25.438975 24817 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:51:25.465655 24817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Generated new TSK 0
I20260927 10:51:25.466421 24817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:51:25.505436 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45913
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:25.639640 24829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:25.639881 24829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:25.639921 24829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:25.643842 24829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:25.643903 24829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:25.644001 24829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:51:25.648964 24829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:25.649931 24829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:25.650918 24829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:25.660244 24834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:25.660267 24837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:25.660216 24835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:25.660425 24829 server_base.cc:1034] running on GCE node
I20260927 10:51:25.660948 24829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:25.661499 24829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:25.662653 24829 hybrid_clock.cc:648] HybridClock initialized: now 1790506285662633 us; error 39 us; skew 500 ppm
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:25.665443 24829 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:51:25.666330 24829 webserver.cc:460] Webserver started at http://127.9.39.129:41357/ using document root <none> and password file <none>
I20260927 10:51:25.666944 24829 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:25.667006 24829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:25.667243 24829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:25.669206 24829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "8809ed3742434326882b54f8a04a0e01"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "eb35665badfbb8f861f6bc1ec562d0af"
server_key_iv: "a1e8589907e8ac1db2a5f5a2abbf9701"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.669739 24829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "8809ed3742434326882b54f8a04a0e01"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "eb35665badfbb8f861f6bc1ec562d0af"
server_key_iv: "a1e8589907e8ac1db2a5f5a2abbf9701"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.673316 24829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:51:25.675650 24844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.676872 24829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:25.677021 24829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "8809ed3742434326882b54f8a04a0e01"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "eb35665badfbb8f861f6bc1ec562d0af"
server_key_iv: "a1e8589907e8ac1db2a5f5a2abbf9701"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.677150 24829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:25.688567 24829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:25.691711 24829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:25.691927 24829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:25.692497 24829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:25.693303 24829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:25.693373 24829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.693444 24829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:25.693488 24829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.702538 24829 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:36551
I20260927 10:51:25.702564 24957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:36551 every 8 connection(s)
I20260927 10:51:25.703512 24829 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:51:25.712270 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24829
I20260927 10:51:25.712409 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:51:25.712765 9374 external_mini_cluster.cc:1428] Setting key c11f4c7187d192d24bdc9634ef48fa85
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:25.717355 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:25.705322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:55881 (local address 127.9.39.190:45913)
0927 10:51:25.705461 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:25.705465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:25.706433 (+ 968us) server_negotiation.cc:407] Connection header received
0927 10:51:25.707371 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:25.707373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:25.707418 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:25.707504 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:25.708623 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.709240 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.710566 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.710881 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.710969 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:25.713655 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:25.713658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:25.713660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:25.713683 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:25.715159 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:25.715798 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:25.715800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:25.715801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:25.715932 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:25.716312 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:25.716317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:25.716320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:25.716560 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:25.716677 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:25.717124 (+ 447us) server_negotiation.cc:299] Negotiation successful
0927 10:51:25.717238 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:51:25.718115 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:25.705683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:55881)
0927 10:51:25.706313 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:25.706351 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:25.707176 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:25.707657 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:25.707663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:25.708026 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:25.708501 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:25.708514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.709347 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.709350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:25.710455 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:25.710465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.711007 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.711010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:25.711136 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:25.711777 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:25.711793 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:25.713482 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:25.715290 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:25.715298 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:25.715317 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:25.715694 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:25.716043 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:25.716048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:25.716051 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:25.716219 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:25.716672 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:25.716676 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:25.716901 (+ 225us) client_negotiation.cc:769] Sending connection context
0927 10:51:25.717049 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 10:51:25.717306 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":110,"threads_started":1}
I20260927 10:51:25.719470 24958 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45913
I20260927 10:51:25.719745 24958 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:25.720376 24958 heartbeater.cc:511] Master 127.9.39.190:45913 requested a full tablet report, sending...
I20260927 10:51:25.721953 24767 ts_manager.cc:194] Registered new tserver with Master: 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551)
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:51:25.723194 24767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:55881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:51:25.739727 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45913
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:25.847919 24963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:25.848168 24963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:25.848209 24963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:25.852218 24963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:25.852269 24963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:25.852365 24963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:51:25.857817 24963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:25.858853 24963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:25.859886 24963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:25.869357 24971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:25.869364 24969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:25.869529 24968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:25.869578 24963 server_base.cc:1034] running on GCE node
I20260927 10:51:25.869944 24963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:25.870644 24963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:25.871855 24963 hybrid_clock.cc:648] HybridClock initialized: now 1790506285871843 us; error 30 us; skew 500 ppm
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:25.874481 24963 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:51:25.875455 24963 webserver.cc:460] Webserver started at http://127.9.39.130:44273/ using document root <none> and password file <none>
I20260927 10:51:25.876017 24963 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:25.876111 24963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:25.876343 24963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:25.878656 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "8299a7dcbc694313a24e18cda0a14eff"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "f4edc260a3b7d9179eb91dce3c1ea9ce"
server_key_iv: "da56b6380c1a1d19f29953349f5b0365"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.879386 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "8299a7dcbc694313a24e18cda0a14eff"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "f4edc260a3b7d9179eb91dce3c1ea9ce"
server_key_iv: "da56b6380c1a1d19f29953349f5b0365"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.883463 24963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 10:51:25.885990 24978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.887342 24963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:25.887485 24963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "8299a7dcbc694313a24e18cda0a14eff"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "f4edc260a3b7d9179eb91dce3c1ea9ce"
server_key_iv: "da56b6380c1a1d19f29953349f5b0365"
server_key_version: "encryptionkey@0"
I20260927 10:51:25.887610 24963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:25.904381 24963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:25.907421 24963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:25.907625 24963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:25.908231 24963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:25.909047 24963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:25.909121 24963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.909194 24963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:25.909255 24963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:25.918478 24963 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:43397
I20260927 10:51:25.918506 25091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:43397 every 8 connection(s)
I20260927 10:51:25.919461 24963 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:51:25.925940 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 24963
I20260927 10:51:25.926030 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:51:25.926307 9374 external_mini_cluster.cc:1428] Setting key dec7e84a899df33db49337e4163483e4
Sep 27 10:51:25 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506285, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:25.932608 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:25.921206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:52435 (local address 127.9.39.190:45913)
0927 10:51:25.921369 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:25.921373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:25.922361 (+ 988us) server_negotiation.cc:407] Connection header received
0927 10:51:25.923311 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:25.923314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:25.923359 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:25.923433 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:25.924531 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.925077 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.926229 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.926527 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.926638 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:25.929037 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:25.929040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:25.929043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:25.929067 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:25.930476 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:25.931064 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:25.931068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:25.931070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:25.931153 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:25.931502 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:25.931505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:25.931506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:25.931692 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:25.931806 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:25.932323 (+ 517us) server_negotiation.cc:299] Negotiation successful
0927 10:51:25.932440 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 10:51:25.933463 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:25.921572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:52435)
0927 10:51:25.922206 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:25.922255 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:25.923113 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:25.923578 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:25.923584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:25.923930 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:25.924406 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:25.924420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.925182 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.925184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:25.926105 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:25.926115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:25.926678 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:25.926682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:25.926885 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:25.927438 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:25.927458 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:25.928872 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:25.930603 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:25.930608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:25.930624 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:25.930959 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:25.931250 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:25.931253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:25.931255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:25.931400 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:25.931806 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:25.931814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:25.932065 (+ 251us) client_negotiation.cc:769] Sending connection context
0927 10:51:25.932267 (+ 202us) client_negotiation.cc:240] Negotiation successful
0927 10:51:25.932586 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":109,"threads_started":1}
I20260927 10:51:25.934643 25092 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45913
I20260927 10:51:25.934908 25092 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:51:25.935436 25092 heartbeater.cc:511] Master 127.9.39.190:45913 requested a full tablet report, sending...
I20260927 10:51:25.936486 24767 ts_manager.cc:194] Registered new tserver with Master: 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
I20260927 10:51:25.937013 24767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:52435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:51:25.953740 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45913
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:26.064637 25097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:26.064920 25097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:26.064965 25097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:26.069259 25097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:26.069316 25097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:26.069375 25097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:51:26.074546 25097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:26.075568 25097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:26.076599 25097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:26.086508 25103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:26.086501 25102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:26.086598 25097 server_base.cc:1034] running on GCE node
W20260927 10:51:26.086519 25105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:26.087224 25097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:26.087857 25097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:26.089071 25097 hybrid_clock.cc:648] HybridClock initialized: now 1790506286089059 us; error 41 us; skew 500 ppm
Sep 27 10:51:26 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:26.092106 25097 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:51:26.092985 25097 webserver.cc:460] Webserver started at http://127.9.39.131:34527/ using document root <none> and password file <none>
I20260927 10:51:26.093575 25097 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:26.093669 25097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:26.093901 25097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:51:26.096118 25097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "9f4a47c18a8440a897610dc3a83c26fe"
format_stamp: "Formatted at 2026-09-27 10:51:26 on dist-test-slave-p9q6"
server_key: "f7f901aee4596290939c2e199ed4e767"
server_key_iv: "2615de49d8b263edd991cb9111e3127b"
server_key_version: "encryptionkey@0"
I20260927 10:51:26.096599 25097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "9f4a47c18a8440a897610dc3a83c26fe"
format_stamp: "Formatted at 2026-09-27 10:51:26 on dist-test-slave-p9q6"
server_key: "f7f901aee4596290939c2e199ed4e767"
server_key_iv: "2615de49d8b263edd991cb9111e3127b"
server_key_version: "encryptionkey@0"
I20260927 10:51:26.100123 25097 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:51:26.102290 25112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:26.103549 25097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:26.103691 25097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "9f4a47c18a8440a897610dc3a83c26fe"
format_stamp: "Formatted at 2026-09-27 10:51:26 on dist-test-slave-p9q6"
server_key: "f7f901aee4596290939c2e199ed4e767"
server_key_iv: "2615de49d8b263edd991cb9111e3127b"
server_key_version: "encryptionkey@0"
I20260927 10:51:26.103799 25097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:26.118310 25097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:26.121687 25097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:26.121894 25097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:26.122455 25097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:26.123288 25097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:26.123363 25097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:26.123433 25097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:26.123467 25097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:26.132344 25097 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:38441
I20260927 10:51:26.132367 25225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:38441 every 8 connection(s)
I20260927 10:51:26.133242 25097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:51:26.140188 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25097
I20260927 10:51:26.140329 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:51:26.140656 9374 external_mini_cluster.cc:1428] Setting key ddd32b84ce7348bab9b60433b4fecd4d
Sep 27 10:51:26 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:26.146569 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.135022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:38839 (local address 127.9.39.190:45913)
0927 10:51:26.135208 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.135212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.136199 (+ 987us) server_negotiation.cc:407] Connection header received
0927 10:51:26.137024 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.137026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.137073 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.137160 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:26.138322 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.138936 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.140084 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.140483 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.140601 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.143164 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:26.143168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:26.143171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:26.143202 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:26.144510 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:26.145036 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:26.145041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:26.145043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:26.145132 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:26.145472 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:26.145477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:26.145478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:26.145662 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:26.145768 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.146281 (+ 513us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.146410 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:51:26.147529 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.135387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:38839)
0927 10:51:26.136027 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.136066 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:26.136832 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:26.137325 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:26.137331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:26.137686 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:26.138179 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.138194 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.139040 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.139042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.139956 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.139966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.140636 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.140639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.140841 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.141477 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:26.141493 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:26.143000 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:26.144628 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:26.144633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:26.144649 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:26.144929 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:26.145228 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:26.145231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:26.145234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:26.145376 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:26.145776 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:26.145785 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:26.146037 (+ 252us) client_negotiation.cc:769] Sending connection context
0927 10:51:26.146242 (+ 205us) client_negotiation.cc:240] Negotiation successful
0927 10:51:26.146567 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":112,"threads_started":1}
I20260927 10:51:26.148658 25226 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45913
I20260927 10:51:26.148893 25226 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:26.149371 25226 heartbeater.cc:511] Master 127.9.39.190:45913 requested a full tablet report, sending...
I20260927 10:51:26.150425 24767 ts_manager.cc:194] Registered new tserver with Master: 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441)
I20260927 10:51:26.151013 24767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:38839
I20260927 10:51:26.154444 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:51:26.164136 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.156503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.9.39.190:45913)
0927 10:51:26.156690 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.156695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.156831 (+ 136us) server_negotiation.cc:407] Connection header received
0927 10:51:26.156978 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.156981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.157030 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.157113 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:26.157831 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.158373 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.159293 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.159518 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.159632 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.160988 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:26.160991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:26.160994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:26.161019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:26.162468 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:26.163007 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:26.163011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:26.163012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:26.163113 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:26.163348 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:26.163350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:26.163351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:26.163548 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:26.163649 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.163838 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.163933 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 10:51:26.167102 24767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:51:26.170055 24767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:51:26.184998 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.179448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57028 (local address 127.9.39.130:43397)
0927 10:51:26.179863 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.179875 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.179899 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:51:26.179975 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.179982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.180161 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.180340 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:26.180620 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.181887 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.183656 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.184227 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.184352 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.184503 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.184723 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.184840 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":166,"threads_started":1}
I20260927 10:51:26.184994 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.178618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:38441 (local address 127.0.0.1:43592)
0927 10:51:26.179005 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.179055 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:26.179178 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:26.180074 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:26.180080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:26.180104 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:26.180259 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.180267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.181837 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.181840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.183668 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.183680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.184448 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.184452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.184585 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.184612 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:51:26.184697 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 10:51:26.184842 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":389,"thread_start_us":114,"threads_started":1}
I20260927 10:51:26.184998 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.178822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43397 (local address 127.0.0.1:57028)
0927 10:51:26.179342 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.179360 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:26.179463 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:26.180307 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:26.180310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:26.180327 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:26.180437 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.180443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.182023 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.182026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.183527 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.183538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.184352 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.184356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.184490 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.184540 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 10:51:26.184676 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 10:51:26.184842 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":141,"thread_start_us":116,"threads_started":1}
I20260927 10:51:26.185598 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.179048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43592 (local address 127.9.39.131:38441)
0927 10:51:26.179511 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.179530 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.179554 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:51:26.179643 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.179652 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.179893 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.180423 (+ 530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:26.180622 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.181708 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.183781 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.184316 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.184435 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.184580 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.185354 (+ 774us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.185450 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":134,"threads_started":1}
I20260927 10:51:26.186398 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.180253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57124 (local address 127.9.39.129:36551)
0927 10:51:26.180649 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.180665 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.180686 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:51:26.180753 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.180758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.180918 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.181062 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:26.181295 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.182306 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.184718 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.185650 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.185751 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.185868 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.186156 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.186251 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":72,"threads_started":1}
I20260927 10:51:26.188246 25160 tablet_service.cc:1467] Processing CreateTablet for tablet d5e9a6e1ec4c4243b88a23947c7c799d (DEFAULT_TABLE table=test-table [id=4b6cb1d644074bd787172547c21ad42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:26.188251 25026 tablet_service.cc:1467] Processing CreateTablet for tablet d5e9a6e1ec4c4243b88a23947c7c799d (DEFAULT_TABLE table=test-table [id=4b6cb1d644074bd787172547c21ad42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:26.189170 25026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e9a6e1ec4c4243b88a23947c7c799d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:26.189172 25160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e9a6e1ec4c4243b88a23947c7c799d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:26.190496 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.179715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36551 (local address 127.0.0.1:57124)
0927 10:51:26.179968 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.179981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:26.180063 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:26.181061 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:26.181064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:26.181082 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:26.181165 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.181171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.182426 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.182431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.184588 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.184598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.185927 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.185931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.186010 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.186030 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:51:26.186106 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 10:51:26.186182 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":1114,"thread_start_us":95,"threads_started":1}
I20260927 10:51:26.193835 24892 tablet_service.cc:1467] Processing CreateTablet for tablet d5e9a6e1ec4c4243b88a23947c7c799d (DEFAULT_TABLE table=test-table [id=4b6cb1d644074bd787172547c21ad42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:26.194913 24892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e9a6e1ec4c4243b88a23947c7c799d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:26.195317 25243 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Bootstrap starting.
I20260927 10:51:26.196664 25244 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Bootstrap starting.
I20260927 10:51:26.197721 25243 tablet_bootstrap.cc:654] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:26.198426 25243 log.cc:824] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:26.199126 25244 tablet_bootstrap.cc:654] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:26.199998 25244 log.cc:824] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:26.200217 25243 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: No bootstrap required, opened a new log
I20260927 10:51:26.200409 25243 ts_tablet_manager.cc:1397] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:51:26.201889 25244 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: No bootstrap required, opened a new log
I20260927 10:51:26.202073 25244 ts_tablet_manager.cc:1397] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260927 10:51:26.203475 25247 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Bootstrap starting.
I20260927 10:51:26.203402 25243 raft_consensus.cc:348] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.203716 25243 raft_consensus.cc:374] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:26.203799 25243 raft_consensus.cc:729] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8299a7dcbc694313a24e18cda0a14eff, State: Initialized, Role: FOLLOWER
I20260927 10:51:26.204274 25243 consensus_queue.cc:260] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.205235 25092 heartbeater.cc:503] Master 127.9.39.190:45913 was elected leader, sending a full tablet report...
I20260927 10:51:26.205360 25243 ts_tablet_manager.cc:1428] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:51:26.205643 25247 tablet_bootstrap.cc:654] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:26.206842 25247 log.cc:824] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:26.206765 25244 raft_consensus.cc:348] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.207027 25244 raft_consensus.cc:374] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:26.207080 25244 raft_consensus.cc:729] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4a47c18a8440a897610dc3a83c26fe, State: Initialized, Role: FOLLOWER
I20260927 10:51:26.207556 25244 consensus_queue.cc:260] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.208194 25226 heartbeater.cc:503] Master 127.9.39.190:45913 was elected leader, sending a full tablet report...
I20260927 10:51:26.208808 25247 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: No bootstrap required, opened a new log
I20260927 10:51:26.208941 25244 ts_tablet_manager.cc:1428] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 10:51:26.208968 25247 ts_tablet_manager.cc:1397] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260927 10:51:26.212438 25247 raft_consensus.cc:348] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.212703 25247 raft_consensus.cc:374] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:26.212759 25247 raft_consensus.cc:729] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8809ed3742434326882b54f8a04a0e01, State: Initialized, Role: FOLLOWER
I20260927 10:51:26.213119 25247 consensus_queue.cc:260] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.213701 24958 heartbeater.cc:503] Master 127.9.39.190:45913 was elected leader, sending a full tablet report...
I20260927 10:51:26.213884 25247 ts_tablet_manager.cc:1428] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260927 10:51:26.384142 25227 tablet.cc:2369] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:26.401273 25251 raft_consensus.cc:482] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:26.401463 25251 raft_consensus.cc:504] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.402558 25251 leader_election.cc:288] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
I20260927 10:51:26.406430 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.402939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:37467 (local address 127.9.39.131:38441)
0927 10:51:26.403087 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.403092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.403218 (+ 126us) server_negotiation.cc:407] Connection header received
0927 10:51:26.403380 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.403383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.403435 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.403522 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:26.403789 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.404288 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.405475 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.405869 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.405994 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.406057 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.406252 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.406315 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:51:26.406447 25252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.402848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:38441 (local address 127.9.39.129:37467)
0927 10:51:26.403117 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.403133 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:26.403275 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:26.403534 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:26.403537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:26.403551 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:26.403684 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.403689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.404391 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.404397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.405361 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.405371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.405989 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.405993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.406125 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.406150 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:51:26.406227 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 10:51:26.406302 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":46,"thread_start_us":104,"threads_started":1}
I20260927 10:51:26.406662 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.402848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43397 (local address 127.9.39.129:40361)
0927 10:51:26.403240 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.403251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:26.403371 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:26.403626 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:26.403629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:26.403644 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:26.403761 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.403767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.404447 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.404453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.405448 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:26.405458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.406093 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.406096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:26.406265 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.406282 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 10:51:26.406364 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 10:51:26.406433 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":115,"thread_start_us":83,"threads_started":1}
I20260927 10:51:26.406641 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.402943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:40361 (local address 127.9.39.130:43397)
0927 10:51:26.403187 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.403192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.403330 (+ 138us) server_negotiation.cc:407] Connection header received
0927 10:51:26.403489 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.403492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.403533 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.403618 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:26.403879 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.404341 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.405566 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.405961 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.406097 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.406153 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.406393 (+ 240us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.406498 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260927 10:51:26.407446 25180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9a6e1ec4c4243b88a23947c7c799d" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4a47c18a8440a897610dc3a83c26fe" is_pre_election: true
I20260927 10:51:26.407490 25046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9a6e1ec4c4243b88a23947c7c799d" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8299a7dcbc694313a24e18cda0a14eff" is_pre_election: true
I20260927 10:51:26.407769 25180 raft_consensus.cc:2454] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8809ed3742434326882b54f8a04a0e01 in term 0.
I20260927 10:51:26.407783 25046 raft_consensus.cc:2454] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8809ed3742434326882b54f8a04a0e01 in term 0.
I20260927 10:51:26.408288 24847 leader_election.cc:302] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01, 9f4a47c18a8440a897610dc3a83c26fe; no voters:
I20260927 10:51:26.408546 25251 raft_consensus.cc:2781] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:51:26.408648 25251 raft_consensus.cc:482] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:51:26.408708 25251 raft_consensus.cc:3037] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:26.409783 25251 raft_consensus.cc:504] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.410189 25251 leader_election.cc:288] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 election: Requested vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
I20260927 10:51:26.410553 25046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9a6e1ec4c4243b88a23947c7c799d" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8299a7dcbc694313a24e18cda0a14eff"
I20260927 10:51:26.410583 25180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9a6e1ec4c4243b88a23947c7c799d" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4a47c18a8440a897610dc3a83c26fe"
I20260927 10:51:26.410666 25046 raft_consensus.cc:3037] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:26.410737 25180 raft_consensus.cc:3037] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:26.411798 25180 raft_consensus.cc:2454] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8809ed3742434326882b54f8a04a0e01 in term 1.
I20260927 10:51:26.411840 25046 raft_consensus.cc:2454] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8809ed3742434326882b54f8a04a0e01 in term 1.
I20260927 10:51:26.412127 24847 leader_election.cc:302] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01, 9f4a47c18a8440a897610dc3a83c26fe; no voters:
I20260927 10:51:26.412309 25251 raft_consensus.cc:2781] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:26.412523 25251 raft_consensus.cc:686] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 LEADER]: Becoming Leader. State: Replica: 8809ed3742434326882b54f8a04a0e01, State: Running, Role: LEADER
I20260927 10:51:26.412854 25251 consensus_queue.cc:237] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:26.416074 24767 catalog_manager.cc:5495] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8809ed3742434326882b54f8a04a0e01 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "8809ed3742434326882b54f8a04a0e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } health_report { overall_health: HEALTHY } } }
W20260927 10:51:26.420504 25093 tablet.cc:2369] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:51:26.454916 24959 tablet.cc:2369] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:26.478901 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.475670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.9.39.129:36551)
0927 10:51:26.475798 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:26.475802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:26.475814 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:51:26.475888 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:26.475890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:26.475947 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:26.476038 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:51:26.476265 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.476761 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.477550 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:26.477770 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:26.477845 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:26.478103 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:51:26.478528 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:51:26.478619 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:26.478671 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 10:51:26.478791 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:51:26.484627 25046 raft_consensus.cc:1261] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Refusing update from remote peer 8809ed3742434326882b54f8a04a0e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:26.484627 25180 raft_consensus.cc:1261] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Refusing update from remote peer 8809ed3742434326882b54f8a04a0e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:26.485337 25254 consensus_queue.cc:1035] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:26.485507 25251 consensus_queue.cc:1035] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:26.498807 25257 mvcc.cc:205] Tried to move back new op lower bound from 7333913749434126336 to 7333913749149700096. Current Snapshot: MvccSnapshot[applied={T|T < 7333913749434126336 or (T in {7333913749434126336})}]
I20260927 10:51:26.498921 25258 mvcc.cc:205] Tried to move back new op lower bound from 7333913749434126336 to 7333913749149700096. Current Snapshot: MvccSnapshot[applied={T|T < 7333913749434126336 or (T in {7333913749434126336})}]
I20260927 10:51:26.500280 24766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:26.500447 24766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:26.502746 24766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Sending DeleteTablet for 3 replicas of tablet d5e9a6e1ec4c4243b88a23947c7c799d
I20260927 10:51:26.503482 25160 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e9a6e1ec4c4243b88a23947c7c799d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:26 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:43592
I20260927 10:51:26.503511 25026 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e9a6e1ec4c4243b88a23947c7c799d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:26 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:57028
I20260927 10:51:26.503989 25267 tablet_replica.cc:330] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: stopping tablet replica
I20260927 10:51:26.504004 25268 tablet_replica.cc:330] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: stopping tablet replica
I20260927 10:51:26.504174 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 24829
I20260927 10:51:26.504316 25267 raft_consensus.cc:2229] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:26.504504 25268 raft_consensus.cc:2229] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:26.504621 25267 raft_consensus.cc:2258] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:26.504959 25268 raft_consensus.cc:2258] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:26.506485 25268 ts_tablet_manager.cc:1905] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:26.506997 25267 ts_tablet_manager.cc:1905] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:26.509166 25267 ts_tablet_manager.cc:1918] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:26.509182 25268 ts_tablet_manager.cc:1918] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:26.509267 25268 log.cc:1196] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/d5e9a6e1ec4c4243b88a23947c7c799d
I20260927 10:51:26.509270 25267 log.cc:1196] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/d5e9a6e1ec4c4243b88a23947c7c799d
I20260927 10:51:26.509569 25268 ts_tablet_manager.cc:1939] T d5e9a6e1ec4c4243b88a23947c7c799d P 8299a7dcbc694313a24e18cda0a14eff: Deleting consensus metadata
I20260927 10:51:26.509663 25267 ts_tablet_manager.cc:1939] T d5e9a6e1ec4c4243b88a23947c7c799d P 9f4a47c18a8440a897610dc3a83c26fe: Deleting consensus metadata
I20260927 10:51:26.510855 24754 catalog_manager.cc:4841] TS 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397): tablet d5e9a6e1ec4c4243b88a23947c7c799d (table test-table [id=4b6cb1d644074bd787172547c21ad42a]) successfully deleted
I20260927 10:51:26.510855 24753 catalog_manager.cc:4841] TS 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441): tablet d5e9a6e1ec4c4243b88a23947c7c799d (table test-table [id=4b6cb1d644074bd787172547c21ad42a]) successfully deleted
W20260927 10:51:26.511762 24752 connection.cc:535] client connection to 127.9.39.129:36551 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 10:51:26.511852 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 24963
W20260927 10:51:26.511950 24752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 10:51:26.517329 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 25097
I20260927 10:51:26.518159 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:26.517592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36551 (local address 127.0.0.1:57144)
0927 10:51:26.517975 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:26.518049 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
W20260927 10:51:26.518323 24752 catalog_manager.cc:4584] TS 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5e9a6e1ec4c4243b88a23947c7c799d: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36551: connect: Connection refused (error 111)
I20260927 10:51:26.522567 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 24735
I20260927 10:51:46.530483 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45913
--webserver_interface=127.9.39.190
--webserver_port=34859
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:51:46.638659 25270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:46.638926 25270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:46.638971 25270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:46.642974 25270 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 10:51:46.643023 25270 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 10:51:46.643039 25270 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 10:51:46.643052 25270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:51:46.643070 25270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:46.643123 25270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:46.643151 25270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:51:46.643183 25270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:51:46.648344 25270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:46.649328 25270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:46.650321 25270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:46.663020 25276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:46.662986 25279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:46.663014 25277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:46.663267 25270 server_base.cc:1034] running on GCE node
I20260927 10:51:46.663918 25270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:46.665071 25270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:46.666230 25270 hybrid_clock.cc:648] HybridClock initialized: now 1790506306666213 us; error 32 us; skew 500 ppm
Sep 27 10:51:46 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506306, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:46.672448 25270 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:51:46.673358 25270 webserver.cc:460] Webserver started at http://127.9.39.190:34859/ using document root <none> and password file <none>
I20260927 10:51:46.673949 25270 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:46.674081 25270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:46.678045 25270 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:51:46.680245 25286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:46.681447 25270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 10:51:46.681586 25270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "bd29ee5fcb234487a5ff59c716bb5036"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "713e73e24e015a86677be964978ae4bf"
server_key_iv: "9dd3f5a1f90d613d515f9f9ecf217cfb"
server_key_version: "encryptionkey@0"
I20260927 10:51:46.682169 25270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:46.697127 25270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:46.700251 25270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:46.700456 25270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:46.707350 25270 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:45913
I20260927 10:51:46.707628 25338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:45913 every 8 connection(s)
I20260927 10:51:46.708499 25270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:51:46.712867 25339 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:51:46.714443 25339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap starting.
I20260927 10:51:46.716809 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25270
I20260927 10:51:46.717232 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:36551
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:45913
--webserver_port=41357
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:51:46.731527 25339 log.cc:824] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:46.740060 25339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:51:46.740482 25339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap complete.
I20260927 10:51:46.744462 25339 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:46.744844 25339 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd29ee5fcb234487a5ff59c716bb5036, State: Initialized, Role: FOLLOWER
I20260927 10:51:46.745323 25339 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:46.745432 25339 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:51:46.745532 25339 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:51:46.745654 25339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 1 FOLLOWER]: Advancing to term 2
I20260927 10:51:46.747519 25339 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:46.747840 25339 leader_election.cc:302] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd29ee5fcb234487a5ff59c716bb5036; no voters:
I20260927 10:51:46.748150 25339 leader_election.cc:288] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 10:51:46.748231 25343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 10:51:46.748431 25343 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 LEADER]: Becoming Leader. State: Replica: bd29ee5fcb234487a5ff59c716bb5036, State: Running, Role: LEADER
I20260927 10:51:46.748713 25343 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:51:46.749413 25339 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:51:46.749598 25343 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd29ee5fcb234487a5ff59c716bb5036. Latest consensus state: current_term: 2 leader_uuid: "bd29ee5fcb234487a5ff59c716bb5036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } } }
I20260927 10:51:46.749691 25343 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:46.750558 25344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bd29ee5fcb234487a5ff59c716bb5036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } } }
I20260927 10:51:46.750656 25344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: This master's current role is: LEADER
I20260927 10:51:46.750887 25347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:51:46.755553 25347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:51:46.756206 25347 catalog_manager.cc:1259] Loaded cluster ID: dfae65ba2ff44766b312fd9ffb58bc2a
I20260927 10:51:46.756259 25347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:51:46.757999 25347 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:51:46.758581 25347 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Loaded TSK: 0
I20260927 10:51:46.769659 25347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Generated new TSK 1
I20260927 10:51:46.769893 25347 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 10:51:46.843068 25341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:46.843307 25341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:46.843346 25341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:46.847357 25341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:46.847414 25341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:46.847508 25341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:51:46.852921 25341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:36551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=41357
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:46.853953 25341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:46.855019 25341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:46.865736 25369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:46.865768 25367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:46.865818 25366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:46.866281 25341 server_base.cc:1034] running on GCE node
I20260927 10:51:46.866773 25341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:46.867559 25341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:46.868760 25341 hybrid_clock.cc:648] HybridClock initialized: now 1790506306868725 us; error 43 us; skew 500 ppm
Sep 27 10:51:46 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506306, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:46.872315 25341 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:51:46.873404 25341 webserver.cc:460] Webserver started at http://127.9.39.129:41357/ using document root <none> and password file <none>
I20260927 10:51:46.874111 25341 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:46.874235 25341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:46.879186 25341 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:51:46.881847 25376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:46.883330 25341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:51:46.883467 25341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "8809ed3742434326882b54f8a04a0e01"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "eb35665badfbb8f861f6bc1ec562d0af"
server_key_iv: "a1e8589907e8ac1db2a5f5a2abbf9701"
server_key_version: "encryptionkey@0"
I20260927 10:51:46.884141 25341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:46.911799 25341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:46.915194 25341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:46.915409 25341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:46.916061 25341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:46.917158 25383 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 10:51:46.921130 25341 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 10:51:46.921214 25341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260927 10:51:46.921298 25341 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 10:51:46.922966 25341 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 10:51:46.923038 25341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:46.923247 25383 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Bootstrap starting.
I20260927 10:51:46.933374 25341 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:36551
I20260927 10:51:46.933401 25490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:36551 every 8 connection(s)
I20260927 10:51:46.934386 25341 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:51:46.943255 25383 log.cc:824] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:46.944156 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25341
I20260927 10:51:46.944628 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:43397
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:45913
--webserver_port=44273
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:51:46 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506306, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:46.952682 25383 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:51:46.953178 25383 tablet_bootstrap.cc:492] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Bootstrap complete.
I20260927 10:51:46.953354 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:46.936647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:46969)
0927 10:51:46.937184 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:46.937242 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:46.938500 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:46.939934 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:46.939943 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:46.940428 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:46.941198 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:46.941220 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.942989 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.942993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:46.944145 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:46.944155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.945184 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.945191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:46.945441 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:46.946179 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:46.946205 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:46.947936 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:46.950348 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:46.950354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:46.950366 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:46.950624 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:46.951000 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:46.951005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:46.951007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:46.951180 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:46.951790 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:46.951796 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:46.952065 (+ 269us) client_negotiation.cc:769] Sending connection context
0927 10:51:46.952248 (+ 183us) client_negotiation.cc:240] Negotiation successful
0927 10:51:46.952542 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":134,"threads_started":1}
I20260927 10:51:46.954715 25383 ts_tablet_manager.cc:1397] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Time spent bootstrapping tablet: real 0.032s user 0.027s sys 0.004s
I20260927 10:51:46.954892 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:46.936772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:46969 (local address 127.9.39.190:45913)
0927 10:51:46.938978 (+ 2206us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:46.938998 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:46.939033 (+ 35us) server_negotiation.cc:407] Connection header received
0927 10:51:46.939236 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:46.939246 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:46.939538 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:46.939798 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:46.941399 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.942643 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.944288 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.945023 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.945197 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:46.948130 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:46.948135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:46.948147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:46.948201 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:46.950226 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:46.950768 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:46.950774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:46.950779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:46.950903 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:46.951269 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:46.951273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:46.951275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:46.951626 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:46.953705 (+ 2079us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:46.953867 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 10:51:46.954128 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2082,"thread_start_us":114,"threads_started":1}
I20260927 10:51:46.956915 25491 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45913
I20260927 10:51:46.957132 25491 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:46.957600 25491 heartbeater.cc:511] Master 127.9.39.190:45913 requested a full tablet report, sending...
I20260927 10:51:46.958389 25383 raft_consensus.cc:348] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:46.958916 25383 raft_consensus.cc:729] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8809ed3742434326882b54f8a04a0e01, State: Initialized, Role: FOLLOWER
I20260927 10:51:46.959426 25383 consensus_queue.cc:260] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:46.960333 25303 ts_manager.cc:194] Registered new tserver with Master: 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551)
I20260927 10:51:46.960438 25383 ts_tablet_manager.cc:1428] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:51:46.963495 25303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:46969
I20260927 10:51:46.965878 25491 heartbeater.cc:503] Master 127.9.39.190:45913 was elected leader, sending a full tablet report...
Sep 27 10:51:46 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506306, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.129@KRBTEST.COM
I20260927 10:51:46.974097 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:46.962931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36551 (local address 127.0.0.1:33176)
0927 10:51:46.963519 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:46.963563 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:46.963740 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:46.964332 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:46.964338 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:46.964790 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:46.964913 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:46.964921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.966378 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.966381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:46.967340 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:46.967351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.968146 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.968149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:46.968233 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:46.968823 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:46.968835 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:46.970422 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:46.972504 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:46.972512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:46.972517 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:46.972738 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:46.973051 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:46.973053 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:46.973055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:46.973129 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:46.973606 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:46.973610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:46.973706 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 10:51:46.973818 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 10:51:46.973960 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":148,"threads_started":1}
I20260927 10:51:46.974241 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:46.962995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.9.39.129:36551)
0927 10:51:46.963266 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:46.963277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:46.963644 (+ 367us) server_negotiation.cc:407] Connection header received
0927 10:51:46.963916 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:46.963922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:46.964136 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:46.964291 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:46.965043 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.966274 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.967470 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:46.968004 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:46.968115 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:46.970563 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:46.970573 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:46.970585 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:46.970611 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:46.972377 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:46.972846 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:46.972851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:46.972853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:46.972943 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:46.973210 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:46.973214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:46.973217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:46.973497 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:46.973628 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:46.973916 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 10:51:46.974088 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260927 10:51:46.975193 25425 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e9a6e1ec4c4243b88a23947c7c799d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:26 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:33176
I20260927 10:51:46.976110 25500 tablet_replica.cc:330] stopping tablet replica
I20260927 10:51:46.976394 25500 raft_consensus.cc:2229] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:46.976637 25500 raft_consensus.cc:2258] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:46.977576 25500 ts_tablet_manager.cc:1905] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:46.981187 25500 ts_tablet_manager.cc:1918] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:46.981294 25500 log.cc:1196] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal/wals/d5e9a6e1ec4c4243b88a23947c7c799d
I20260927 10:51:46.981637 25500 ts_tablet_manager.cc:1939] T d5e9a6e1ec4c4243b88a23947c7c799d P 8809ed3742434326882b54f8a04a0e01: Deleting consensus metadata
I20260927 10:51:46.983316 25288 catalog_manager.cc:4841] TS 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551): tablet d5e9a6e1ec4c4243b88a23947c7c799d (table test-table [id=4b6cb1d644074bd787172547c21ad42a]) successfully deleted
W20260927 10:51:47.064836 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:47.065117 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:47.065162 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:47.069379 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:47.069456 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:47.069522 25496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:51:47.074983 25496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:43397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=44273
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:47.076041 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:47.077024 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:47.086736 25506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:47.086844 25496 server_base.cc:1034] running on GCE node
W20260927 10:51:47.086736 25505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:47.086783 25508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:47.087375 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:47.088047 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:47.089208 25496 hybrid_clock.cc:648] HybridClock initialized: now 1790506307089173 us; error 47 us; skew 500 ppm
Sep 27 10:51:47 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506307, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:47.092011 25496 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:51:47.092864 25496 webserver.cc:460] Webserver started at http://127.9.39.130:44273/ using document root <none> and password file <none>
I20260927 10:51:47.093382 25496 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:47.093453 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:47.097347 25496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:51:47.099339 25515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:47.100543 25496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:51:47.100675 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "8299a7dcbc694313a24e18cda0a14eff"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "f4edc260a3b7d9179eb91dce3c1ea9ce"
server_key_iv: "da56b6380c1a1d19f29953349f5b0365"
server_key_version: "encryptionkey@0"
I20260927 10:51:47.101221 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:47.113233 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:47.116161 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:47.116364 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:47.116962 25496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:47.117775 25496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:47.117844 25496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:47.117913 25496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:47.117965 25496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:47.127094 25496 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:43397
I20260927 10:51:47.127140 25628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:43397 every 8 connection(s)
I20260927 10:51:47.128093 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:51:47.133503 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25496
I20260927 10:51:47.133976 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:38441
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:45913
--webserver_port=34527
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:51:47 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506307, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:47.142836 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.130071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:57815 (local address 127.9.39.190:45913)
0927 10:51:47.130223 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.130227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.131213 (+ 986us) server_negotiation.cc:407] Connection header received
0927 10:51:47.132136 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.132140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.132206 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.132317 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:47.133519 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.134193 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.135443 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.135706 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.135825 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.138597 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:47.138603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:47.138608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:47.138645 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:47.140496 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.141063 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.141069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.141073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.141172 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.141508 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.141512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.141515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.141778 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:47.141896 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.142415 (+ 519us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.142575 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:51:47.143685 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.130372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:57815)
0927 10:51:47.131049 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.131089 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.131920 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.132502 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.132510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.132869 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:47.133355 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.133369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.134332 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.134341 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.135309 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.135319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.135866 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.135870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.136049 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.136655 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:47.136671 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:47.138398 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:47.140639 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:47.140644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:47.140656 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:47.140952 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:47.141273 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:47.141275 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:47.141277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:47.141413 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:47.141927 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:47.141933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:47.142170 (+ 237us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.142332 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.142599 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":108,"threads_started":1}
I20260927 10:51:47.145167 25629 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45913
I20260927 10:51:47.145447 25629 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:47.145958 25629 heartbeater.cc:511] Master 127.9.39.190:45913 requested a full tablet report, sending...
I20260927 10:51:47.147040 25303 ts_manager.cc:194] Registered new tserver with Master: 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
I20260927 10:51:47.147631 25303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:57815
W20260927 10:51:47.246881 25632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:51:47.247164 25632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:51:47.247206 25632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:51:47.251744 25632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:51:47.251807 25632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:51:47.251868 25632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:51:47.257463 25632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:38441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=34527
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:51:47.258526 25632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:51:47.259666 25632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:51:47.269265 25640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:51:47.269304 25637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:47.269523 25632 server_base.cc:1034] running on GCE node
W20260927 10:51:47.269265 25638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:51:47.269933 25632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:51:47.270622 25632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:51:47.271845 25632 hybrid_clock.cc:648] HybridClock initialized: now 1790506307271828 us; error 27 us; skew 500 ppm
Sep 27 10:51:47 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506307, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:51:47.274533 25632 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:51:47.275460 25632 webserver.cc:460] Webserver started at http://127.9.39.131:34527/ using document root <none> and password file <none>
I20260927 10:51:47.275961 25632 fs_manager.cc:359] Metadata directory not provided
I20260927 10:51:47.276043 25632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:51:47.279790 25632 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:51:47.281793 25647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:47.283155 25632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:51:47.283275 25632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "9f4a47c18a8440a897610dc3a83c26fe"
format_stamp: "Formatted at 2026-09-27 10:51:26 on dist-test-slave-p9q6"
server_key: "f7f901aee4596290939c2e199ed4e767"
server_key_iv: "2615de49d8b263edd991cb9111e3127b"
server_key_version: "encryptionkey@0"
I20260927 10:51:47.283849 25632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:51:47.301959 25632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:51:47.305567 25632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:51:47.305776 25632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:51:47.306371 25632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:51:47.307237 25632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:51:47.307284 25632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:47.307350 25632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:51:47.307387 25632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:51:47.316039 25632 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:38441
I20260927 10:51:47.316057 25760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:38441 every 8 connection(s)
I20260927 10:51:47.317044 25632 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:51:47.320472 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25632
Sep 27 10:51:47 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506307, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:51:47.329794 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.321146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.9.39.190:45913)
0927 10:51:47.321381 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.321385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.321672 (+ 287us) server_negotiation.cc:407] Connection header received
0927 10:51:47.321788 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.321791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.321839 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.321937 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:47.322602 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.323295 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.324264 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.324640 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.324743 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.326299 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:47.326303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:47.326308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:47.326335 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:47.328076 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.328567 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.328570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.328571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.328657 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.328932 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.328935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.328937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.329119 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:47.329337 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.329493 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.329620 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260927 10:51:47.332207 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.318893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:55025 (local address 127.9.39.190:45913)
0927 10:51:47.319061 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.319066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.320207 (+ 1141us) server_negotiation.cc:407] Connection header received
0927 10:51:47.321360 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.321364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.321419 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.321533 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:47.322999 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.323585 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.325030 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.325317 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.325421 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.328374 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:47.328378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:47.328381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:47.328411 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:47.329967 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.330570 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.330573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.330574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.330675 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.331097 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.331101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.331102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.331299 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:47.331394 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.332007 (+ 613us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.332102 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 10:51:47.333073 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.319369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:55025)
0927 10:51:47.320048 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.320097 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.321101 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.321713 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.321722 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.322225 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:51:47.322819 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.322839 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.323708 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.323710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.324896 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.324904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.325482 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.325486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.325628 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.326186 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:51:47.326202 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:51:47.328192 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:51:47.330076 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:47.330082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:47.330097 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:47.330465 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:47.330792 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:51:47.330798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:51:47.330800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:51:47.330990 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:51:47.331397 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:51:47.331401 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:51:47.331732 (+ 331us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.331919 (+ 187us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.332173 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":111,"threads_started":1}
I20260927 10:51:47.334198 25761 heartbeater.cc:347] Connected to a master server at 127.9.39.190:45913
I20260927 10:51:47.334406 25761 heartbeater.cc:464] Registering TS with master...
I20260927 10:51:47.334897 25761 heartbeater.cc:511] Master 127.9.39.190:45913 requested a full tablet report, sending...
I20260927 10:51:47.335820 25303 ts_manager.cc:194] Registered new tserver with Master: 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441)
I20260927 10:51:47.336371 25303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:55025
I20260927 10:51:47.343487 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:51:47.347721 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.344105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.9.39.190:45913)
0927 10:51:47.344242 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.344246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.344445 (+ 199us) server_negotiation.cc:407] Connection header received
0927 10:51:47.344594 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.344597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.344706 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.344797 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:51:47.345041 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.345528 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.346469 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.346778 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.346879 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.347236 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:51:47.347369 (+ 133us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 10:51:47.347578 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43270: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260927 10:51:47.347798 25494 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43270: token expired
W20260927 10:51:47.348120 25232 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.39.190:45913)
I20260927 10:51:47.355450 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.348507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.9.39.190:45913)
0927 10:51:47.348624 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.348627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.348639 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:51:47.348721 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.348723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.348755 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.348817 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:51:47.349477 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.349991 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.350834 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.351049 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.351171 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.352431 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:51:47.352434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:51:47.352436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:51:47.352457 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:51:47.353756 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.354194 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.354196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.354198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.354269 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:51:47.354631 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:51:47.354634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:51:47.354636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:51:47.354819 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:51:47.354921 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.355167 (+ 246us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.355253 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:51:47.357808 25303 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:51:47.361169 25303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:51:47.368872 25425 tablet_service.cc:1467] Processing CreateTablet for tablet 4db40513720a42cca682226fb56d68ac (DEFAULT_TABLE table=test-table [id=e0861a4b5a884a33b1e7fc2f98f85339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:47.369516 25425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db40513720a42cca682226fb56d68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:47.372299 25383 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Bootstrap starting.
I20260927 10:51:47.373185 25383 tablet_bootstrap.cc:654] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:47.373615 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.367941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43397 (local address 127.0.0.1:37992)
0927 10:51:47.368376 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.368389 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.368487 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.369046 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.369049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.369071 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:47.369155 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.369162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.370728 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.370735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.371886 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.371898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.373080 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.373087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.373210 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.373262 (+ 52us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.373352 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.373428 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260927 10:51:47.373780 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.367590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:38441 (local address 127.0.0.1:34450)
0927 10:51:47.367744 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.367758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.367836 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.368659 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.368662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.368679 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:47.368779 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.368786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.370275 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.370279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.371347 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.371356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.373370 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.373373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.373479 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.373497 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.373584 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.373661 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":55}
I20260927 10:51:47.373847 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.367779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.9.39.131:38441)
0927 10:51:47.368144 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.368158 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.368183 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:51:47.368266 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.368274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.368510 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.368679 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:47.368877 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.370146 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.371482 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.372126 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.372234 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.372374 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.373620 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.373716 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":121,"threads_started":1}
I20260927 10:51:47.374043 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.368187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.9.39.130:43397)
0927 10:51:47.368588 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.368603 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.368626 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:51:47.368698 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.368703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.368892 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.369053 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:47.369270 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.370578 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.372021 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.372978 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.373129 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.373242 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.373782 (+ 540us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.373885 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":99,"threads_started":1}
I20260927 10:51:47.374773 25383 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: No bootstrap required, opened a new log
I20260927 10:51:47.374869 25383 ts_tablet_manager.cc:1397] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 10:51:47.375308 25383 raft_consensus.cc:348] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.375459 25383 raft_consensus.cc:374] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:47.375509 25383 raft_consensus.cc:729] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8809ed3742434326882b54f8a04a0e01, State: Initialized, Role: FOLLOWER
I20260927 10:51:47.375630 25383 consensus_queue.cc:260] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.375885 25383 ts_tablet_manager.cc:1428] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 10:51:47.377058 25695 tablet_service.cc:1467] Processing CreateTablet for tablet 4db40513720a42cca682226fb56d68ac (DEFAULT_TABLE table=test-table [id=e0861a4b5a884a33b1e7fc2f98f85339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:47.377064 25563 tablet_service.cc:1467] Processing CreateTablet for tablet 4db40513720a42cca682226fb56d68ac (DEFAULT_TABLE table=test-table [id=e0861a4b5a884a33b1e7fc2f98f85339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:51:47.378149 25563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db40513720a42cca682226fb56d68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:47.378157 25695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db40513720a42cca682226fb56d68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:51:47.386185 25774 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Bootstrap starting.
I20260927 10:51:47.386720 25775 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Bootstrap starting.
I20260927 10:51:47.388914 25774 tablet_bootstrap.cc:654] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:47.389379 25775 tablet_bootstrap.cc:654] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Neither blocks nor log segments found. Creating new log.
I20260927 10:51:47.390232 25775 log.cc:824] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:47.390232 25774 log.cc:824] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Log is configured to *not* fsync() on all Append() calls
I20260927 10:51:47.392074 25775 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: No bootstrap required, opened a new log
I20260927 10:51:47.392079 25774 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: No bootstrap required, opened a new log
I20260927 10:51:47.392273 25775 ts_tablet_manager.cc:1397] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:51:47.392293 25774 ts_tablet_manager.cc:1397] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:51:47.395401 25775 raft_consensus.cc:348] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.395401 25774 raft_consensus.cc:348] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.395648 25775 raft_consensus.cc:374] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:47.395641 25774 raft_consensus.cc:374] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:51:47.395727 25774 raft_consensus.cc:729] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4a47c18a8440a897610dc3a83c26fe, State: Initialized, Role: FOLLOWER
I20260927 10:51:47.395728 25775 raft_consensus.cc:729] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8299a7dcbc694313a24e18cda0a14eff, State: Initialized, Role: FOLLOWER
I20260927 10:51:47.396189 25775 consensus_queue.cc:260] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.396190 25774 consensus_queue.cc:260] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.396842 25629 heartbeater.cc:503] Master 127.9.39.190:45913 was elected leader, sending a full tablet report...
I20260927 10:51:47.397143 25774 ts_tablet_manager.cc:1428] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:51:47.397171 25775 ts_tablet_manager.cc:1428] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:51:47.397919 25761 heartbeater.cc:503] Master 127.9.39.190:45913 was elected leader, sending a full tablet report...
W20260927 10:51:47.435788 25492 tablet.cc:2369] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:51:47.568010 25762 tablet.cc:2369] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 10:51:47.629452 25630 tablet.cc:2369] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:51:47.690975 25779 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:47.691316 25497 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:51:47.691272 25779 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.691516 25497 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.692636 25497 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
I20260927 10:51:47.692943 25779 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397), 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551)
I20260927 10:51:47.697402 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.692893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43397 (local address 127.9.39.131:47803)
0927 10:51:47.693229 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.693244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.693343 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.693628 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.693630 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.693646 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:47.693764 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.693774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.694855 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.694858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.696054 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.696064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.696981 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.696985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.697095 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.697117 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.697186 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.697253 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260927 10:51:47.697849 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.693829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:49837 (local address 127.9.39.131:38441)
0927 10:51:47.693934 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.693939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.693952 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:51:47.693996 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.693998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.694050 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.694129 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:47.694473 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.695112 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.696655 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.697286 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.697426 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.697470 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.697664 (+ 194us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.697724 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":37,"spinlock_wait_cycles":8064}
I20260927 10:51:47.697896 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.692901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43397 (local address 127.9.39.129:54547)
0927 10:51:47.693395 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.693407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.693494 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.694176 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.694179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.694200 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:47.694346 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.694353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.695448 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.695452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.696627 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.696636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.697473 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.697477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.697592 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.697622 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.697678 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.697744 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":24,"spinlock_wait_cycles":23936,"thread_start_us":79,"threads_started":1}
I20260927 10:51:47.698026 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.693248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:47803 (local address 127.9.39.130:43397)
0927 10:51:47.693366 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.693373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.693403 (+ 30us) server_negotiation.cc:407] Connection header received
0927 10:51:47.693476 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.693479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.693529 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.693609 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:47.693948 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.694744 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.696188 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.696754 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.696852 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.696911 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.697822 (+ 911us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.697889 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":52}
I20260927 10:51:47.698256 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.692893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:38441 (local address 127.9.39.129:49837)
0927 10:51:47.693260 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.693274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.693383 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.694220 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.694222 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.694287 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:47.694354 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.694359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.695231 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.695235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.696520 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.696529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.697416 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.697420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.697509 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.697556 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.697983 (+ 427us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.698121 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":96,"spinlock_wait_cycles":768,"thread_start_us":90,"threads_started":1}
I20260927 10:51:47.698416 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.693674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:54547 (local address 127.9.39.130:43397)
0927 10:51:47.693959 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.693964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.693976 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:51:47.694011 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.694015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.694050 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.694465 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:47.694563 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.695308 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.696754 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.697330 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.698161 (+ 831us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.698205 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.698239 (+ 34us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.698297 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":65,"threads_started":1}
I20260927 10:51:47.698577 25583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "9f4a47c18a8440a897610dc3a83c26fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8299a7dcbc694313a24e18cda0a14eff" is_pre_election: true
I20260927 10:51:47.698819 25582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8299a7dcbc694313a24e18cda0a14eff" is_pre_election: true
I20260927 10:51:47.698942 25583 raft_consensus.cc:2454] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f4a47c18a8440a897610dc3a83c26fe in term 0.
I20260927 10:51:47.698961 25715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4a47c18a8440a897610dc3a83c26fe" is_pre_election: true
I20260927 10:51:47.699139 25715 raft_consensus.cc:2454] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8809ed3742434326882b54f8a04a0e01 in term 0.
I20260927 10:51:47.699427 25651 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe; no voters:
I20260927 10:51:47.699712 25779 raft_consensus.cc:2781] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:51:47.699716 25379 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01, 9f4a47c18a8440a897610dc3a83c26fe; no voters: 8299a7dcbc694313a24e18cda0a14eff
I20260927 10:51:47.699800 25779 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:51:47.699853 25779 raft_consensus.cc:3037] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:47.699865 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.692901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36551 (local address 127.9.39.131:47475)
0927 10:51:47.693922 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.693936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:51:47.694031 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:51:47.694349 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:51:47.694352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:51:47.694367 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:51:47.695234 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.695240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.696243 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.696246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.697788 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:51:47.697795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.699475 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.699479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:51:47.699571 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.699593 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:51:47.699660 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 10:51:47.699722 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":690,"spinlock_wait_cycles":2304,"thread_start_us":104,"threads_started":1}
I20260927 10:51:47.700115 25497 raft_consensus.cc:2781] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:51:47.700246 25497 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:51:47.700265 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.693166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:47475 (local address 127.9.39.129:36551)
0927 10:51:47.693546 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.693551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.694106 (+ 555us) server_negotiation.cc:407] Connection header received
0927 10:51:47.694148 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.694151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.694190 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.694261 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:51:47.695333 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.696116 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.697910 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.698597 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.699625 (+ 1028us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.699745 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.700029 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.700121 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":311,"thread_start_us":66,"threads_started":1}
I20260927 10:51:47.700322 25497 raft_consensus.cc:3037] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:47.700727 25445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "9f4a47c18a8440a897610dc3a83c26fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8809ed3742434326882b54f8a04a0e01" is_pre_election: true
I20260927 10:51:47.701086 25779 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.701419 25779 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [CANDIDATE]: Term 1 election: Requested vote from peers 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397), 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551)
I20260927 10:51:47.701648 25497 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.701720 25582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "9f4a47c18a8440a897610dc3a83c26fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8299a7dcbc694313a24e18cda0a14eff"
I20260927 10:51:47.701815 25444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "9f4a47c18a8440a897610dc3a83c26fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8809ed3742434326882b54f8a04a0e01"
I20260927 10:51:47.701823 25582 raft_consensus.cc:3037] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:51:47.702020 25497 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 election: Requested vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
I20260927 10:51:47.702195 25445 raft_consensus.cc:2379] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f4a47c18a8440a897610dc3a83c26fe in current term 1: Already voted for candidate 8809ed3742434326882b54f8a04a0e01 in this term.
I20260927 10:51:47.702450 25583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8299a7dcbc694313a24e18cda0a14eff"
I20260927 10:51:47.703081 25715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db40513720a42cca682226fb56d68ac" candidate_uuid: "8809ed3742434326882b54f8a04a0e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4a47c18a8440a897610dc3a83c26fe"
I20260927 10:51:47.703231 25715 raft_consensus.cc:2379] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8809ed3742434326882b54f8a04a0e01 in current term 1: Already voted for candidate 9f4a47c18a8440a897610dc3a83c26fe in this term.
I20260927 10:51:47.703253 25582 raft_consensus.cc:2454] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f4a47c18a8440a897610dc3a83c26fe in term 1.
I20260927 10:51:47.703490 25651 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe; no voters: 8809ed3742434326882b54f8a04a0e01
I20260927 10:51:47.703636 25379 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01; no voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe
I20260927 10:51:47.703699 25779 raft_consensus.cc:2781] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:51:47.703819 25497 raft_consensus.cc:2726] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 10:51:47.703938 25779 raft_consensus.cc:686] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 1 LEADER]: Becoming Leader. State: Replica: 9f4a47c18a8440a897610dc3a83c26fe, State: Running, Role: LEADER
I20260927 10:51:47.704252 25779 consensus_queue.cc:237] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:51:47.707572 25303 catalog_manager.cc:5495] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131). New cstate: current_term: 1 leader_uuid: "9f4a47c18a8440a897610dc3a83c26fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } health_report { overall_health: UNKNOWN } } }
I20260927 10:51:47.740231 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.736818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.9.39.131:38441)
0927 10:51:47.736956 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:51:47.736959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:51:47.737010 (+ 51us) server_negotiation.cc:407] Connection header received
0927 10:51:47.737088 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:51:47.737090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:51:47.737143 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:51:47.737233 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:51:47.737497 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.737996 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.738824 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:51:47.739074 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:51:47.739150 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:51:47.739415 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:51:47.739836 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:51:47.739925 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 10:51:47.740011 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 10:51:47.740054 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:51:47.746344 25445 raft_consensus.cc:1261] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Refusing update from remote peer 9f4a47c18a8440a897610dc3a83c26fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:47.746333 25582 raft_consensus.cc:1261] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Refusing update from remote peer 9f4a47c18a8440a897610dc3a83c26fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:51:47.747002 25785 consensus_queue.cc:1035] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:47.747210 25779 consensus_queue.cc:1035] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:51:47.755308 25788 mvcc.cc:205] Tried to move back new op lower bound from 7333913836520579072 to 7333913836359094272. Current Snapshot: MvccSnapshot[applied={T|T < 7333913836520579072 or (T in {7333913836520579072})}]
I20260927 10:51:47.761989 25303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:47.762161 25303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:51:47.764369 25303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Sending DeleteTablet for 3 replicas of tablet 4db40513720a42cca682226fb56d68ac
I20260927 10:51:47.765000 25695 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db40513720a42cca682226fb56d68ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:47 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34450
I20260927 10:51:47.765081 25563 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db40513720a42cca682226fb56d68ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:47 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:37992
I20260927 10:51:47.765475 25798 tablet_replica.cc:330] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: stopping tablet replica
I20260927 10:51:47.765656 25799 tablet_replica.cc:330] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: stopping tablet replica
I20260927 10:51:47.765663 25425 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db40513720a42cca682226fb56d68ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:51:47 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:33176
I20260927 10:51:47.765918 25798 raft_consensus.cc:2229] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:51:47.765952 25799 raft_consensus.cc:2229] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:47.766141 25800 tablet_replica.cc:330] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: stopping tablet replica
I20260927 10:51:47.766160 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 25341
I20260927 10:51:47.766261 25800 raft_consensus.cc:2229] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:51:47.766270 25799 raft_consensus.cc:2258] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:47.766386 25800 raft_consensus.cc:2258] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:47.766505 25798 raft_consensus.cc:2258] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:51:47.767902 25798 ts_tablet_manager.cc:1905] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:47.768975 25799 ts_tablet_manager.cc:1905] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:51:47.770057 25798 ts_tablet_manager.cc:1918] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:47.770169 25798 log.cc:1196] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/4db40513720a42cca682226fb56d68ac
I20260927 10:51:47.770467 25798 ts_tablet_manager.cc:1939] T 4db40513720a42cca682226fb56d68ac P 9f4a47c18a8440a897610dc3a83c26fe: Deleting consensus metadata
I20260927 10:51:47.771721 25289 catalog_manager.cc:4841] TS 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441): tablet 4db40513720a42cca682226fb56d68ac (table test-table [id=e0861a4b5a884a33b1e7fc2f98f85339]) successfully deleted
I20260927 10:51:47.772333 25799 ts_tablet_manager.cc:1918] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:51:47.772435 25799 log.cc:1196] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/4db40513720a42cca682226fb56d68ac
I20260927 10:51:47.772886 25799 ts_tablet_manager.cc:1939] T 4db40513720a42cca682226fb56d68ac P 8299a7dcbc694313a24e18cda0a14eff: Deleting consensus metadata
I20260927 10:51:47.774168 25290 catalog_manager.cc:4841] TS 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397): tablet 4db40513720a42cca682226fb56d68ac (table test-table [id=e0861a4b5a884a33b1e7fc2f98f85339]) successfully deleted
W20260927 10:51:47.775105 25288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:36551 (error 108)
I20260927 10:51:47.776109 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:51:47.775798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:36551 (local address 127.0.0.1:33190)
0927 10:51:47.775925 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 10:51:47.776009 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 10:51:47.776275 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 25496
W20260927 10:51:47.776317 25288 catalog_manager.cc:4584] TS 8809ed3742434326882b54f8a04a0e01 (127.9.39.129:36551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4db40513720a42cca682226fb56d68ac: Network error: Client connection negotiation failed: client connection to 127.9.39.129:36551: connect: Connection refused (error 111)
I20260927 10:51:47.783066 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 25632
I20260927 10:51:47.789883 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 25270
I20260927 10:52:07.796303 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45913
--webserver_interface=127.9.39.190
--webserver_port=34859
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:07.904577 25802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:07.904872 25802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:07.904951 25802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:07.908990 25802 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 10:52:07.909058 25802 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 10:52:07.909085 25802 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 10:52:07.909116 25802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:07.909157 25802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:07.909195 25802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:07.909260 25802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:07.909298 25802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:07.914407 25802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:07.915467 25802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:07.916553 25802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:07.924978 25808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:07.925102 25802 server_base.cc:1034] running on GCE node
W20260927 10:52:07.925017 25810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:07.925017 25807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:07.925766 25802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:07.926888 25802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:07.928083 25802 hybrid_clock.cc:648] HybridClock initialized: now 1790506327928064 us; error 30 us; skew 500 ppm
Sep 27 10:52:07 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506327, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:07.937866 25802 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:07.938934 25802 webserver.cc:460] Webserver started at http://127.9.39.190:34859/ using document root <none> and password file <none>
I20260927 10:52:07.939492 25802 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:07.939553 25802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:07.943405 25802 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 10:52:07.945505 25817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:07.946846 25802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:07.946982 25802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "bd29ee5fcb234487a5ff59c716bb5036"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "713e73e24e015a86677be964978ae4bf"
server_key_iv: "9dd3f5a1f90d613d515f9f9ecf217cfb"
server_key_version: "encryptionkey@0"
I20260927 10:52:07.947539 25802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:07.976503 25802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:07.979864 25802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:07.980069 25802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:07.987159 25802 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:45913
I20260927 10:52:07.987470 25869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:45913 every 8 connection(s)
I20260927 10:52:07.988230 25802 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:07.992254 25870 sys_catalog.cc:263] Verifying existing consensus state
I20260927 10:52:07.993206 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25802
I20260927 10:52:07.993644 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:36551
--local_ip_for_outbound_sockets=127.9.39.129
--tserver_master_addrs=127.9.39.190:45913
--webserver_port=41357
--webserver_interface=127.9.39.129
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:52:07.993979 25870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap starting.
I20260927 10:52:08.011206 25870 log.cc:824] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:08.022153 25870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:52:08.022550 25870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Bootstrap complete.
I20260927 10:52:08.026440 25870 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:52:08.026868 25870 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd29ee5fcb234487a5ff59c716bb5036, State: Initialized, Role: FOLLOWER
I20260927 10:52:08.027328 25870 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:52:08.027436 25870 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:08.027554 25870 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:08.027669 25870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 2 FOLLOWER]: Advancing to term 3
I20260927 10:52:08.029528 25870 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:52:08.029846 25870 leader_election.cc:302] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd29ee5fcb234487a5ff59c716bb5036; no voters:
I20260927 10:52:08.030180 25870 leader_election.cc:288] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 10:52:08.030256 25874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 10:52:08.030455 25874 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [term 3 LEADER]: Becoming Leader. State: Replica: bd29ee5fcb234487a5ff59c716bb5036, State: Running, Role: LEADER
I20260927 10:52:08.030800 25874 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } }
I20260927 10:52:08.031415 25870 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:08.031699 25874 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd29ee5fcb234487a5ff59c716bb5036. Latest consensus state: current_term: 3 leader_uuid: "bd29ee5fcb234487a5ff59c716bb5036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } } }
I20260927 10:52:08.031817 25874 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:08.032187 25875 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bd29ee5fcb234487a5ff59c716bb5036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd29ee5fcb234487a5ff59c716bb5036" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 45913 } } }
I20260927 10:52:08.032339 25875 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:08.033483 25878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:08.038195 25878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:08.038872 25878 catalog_manager.cc:1259] Loaded cluster ID: dfae65ba2ff44766b312fd9ffb58bc2a
I20260927 10:52:08.038930 25878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:08.041352 25878 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:08.042071 25878 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Loaded TSK: 0
I20260927 10:52:08.042153 25878 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd29ee5fcb234487a5ff59c716bb5036: Loaded TSK: 1
F20260927 10:52:08.042940 25878 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1ea457f9ad google::LogMessage::Fail() at ??:0
@ 0x7f1ea458288a google::LogMessage::SendToLog() at ??:0
@ 0x7f1ea457f4de google::LogMessage::Flush() at ??:0
@ 0x7f1ea4580a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1ea724318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1ea723bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1eaeee47bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1eaeebe99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1eaeebf178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f1eaef00df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1eaef1f82b std::function<>::operator()() at ??:0
@ 0x7f1eaeebeb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1eaeebff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1eaeebc579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1eaef00640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1ea67d13d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1ea67d1c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1ea67d33f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1ea67c2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1ea7e066db start_thread at ??:0
@ 0x7f1ea307d61f clone at ??:0
W20260927 10:52:08.140271 25872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:08.140524 25872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:08.140568 25872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:08.144729 25872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:08.144812 25872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:08.144927 25872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:08.150238 25872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:36551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=41357
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:08.151530 25872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:08.152984 25872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:08.164134 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:08.164203 25918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:08.164363 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:08.164665 25872 server_base.cc:1034] running on GCE node
I20260927 10:52:08.165110 25872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:08.165830 25872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:08.167013 25872 hybrid_clock.cc:648] HybridClock initialized: now 1790506328166978 us; error 50 us; skew 500 ppm
Sep 27 10:52:08 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:08.170126 25872 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:52:08.171236 25872 webserver.cc:460] Webserver started at http://127.9.39.129:41357/ using document root <none> and password file <none>
I20260927 10:52:08.171921 25872 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:08.172010 25872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:08.176968 25872 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:52:08.179585 25927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.181098 25872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:52:08.181210 25872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "8809ed3742434326882b54f8a04a0e01"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "eb35665badfbb8f861f6bc1ec562d0af"
server_key_iv: "a1e8589907e8ac1db2a5f5a2abbf9701"
server_key_version: "encryptionkey@0"
I20260927 10:52:08.181877 25872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:08.238655 25872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:08.242218 25872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:08.242435 25872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:08.243109 25872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:08.244210 25934 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 10:52:08.249578 25872 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 10:52:08.249655 25872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260927 10:52:08.249758 25872 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 10:52:08.251494 25872 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 10:52:08.251571 25872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:08.251859 25934 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Bootstrap starting.
I20260927 10:52:08.262094 25872 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:36551
I20260927 10:52:08.262118 26041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:36551 every 8 connection(s)
I20260927 10:52:08.263063 25872 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:08.271270 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 25872
I20260927 10:52:08.271817 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:43397
--local_ip_for_outbound_sockets=127.9.39.130
--tserver_master_addrs=127.9.39.190:45913
--webserver_port=44273
--webserver_interface=127.9.39.130
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:52:08.273236 25934 log.cc:824] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Log is configured to *not* fsync() on all Append() calls
W20260927 10:52:08.278142 26044 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:52:08.265053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:39597)
0927 10:52:08.265518 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.265568 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:08.266361 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:08.277383 (+ 11022us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":139,"threads_started":1}
W20260927 10:52:08.278666 26042 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:45913 (0 consecutive failures): Network error: Failed to ping master at 127.9.39.190:45913: Client connection negotiation failed: client connection to 127.9.39.190:45913: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 10:52:08.279373 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.279151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:55943)
0927 10:52:08.279265 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.279325 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 10:52:08.280061 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.279839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:36057)
0927 10:52:08.279971 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.280017 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260927 10:52:08.280254 26042 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 10:52:08.282694 25934 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 10:52:08.283160 25934 tablet_bootstrap.cc:492] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Bootstrap complete.
I20260927 10:52:08.284652 25934 ts_tablet_manager.cc:1397] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.016s
I20260927 10:52:08.288261 25934 raft_consensus.cc:348] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:52:08.288784 25934 raft_consensus.cc:729] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8809ed3742434326882b54f8a04a0e01, State: Initialized, Role: FOLLOWER
I20260927 10:52:08.289270 25934 consensus_queue.cc:260] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:52:08.290262 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.290049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:52009)
0927 10:52:08.290157 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.290209 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 10:52:08.290242 25934 ts_tablet_manager.cc:1428] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260927 10:52:08.291034 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.290834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:45379)
0927 10:52:08.290939 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.290982 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 10:52:08.389505 26045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:08.389755 26045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:08.389833 26045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:08.393944 26045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:08.394001 26045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:08.394112 26045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:08.399223 26045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:43397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=44273
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:08.400202 26045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:08.401238 26045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:52:08.410745 26045 server_base.cc:1034] running on GCE node
W20260927 10:52:08.410660 26053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:08.410660 26055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:08.410748 26052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:08.411384 26045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:08.411974 26045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:08.413154 26045 hybrid_clock.cc:648] HybridClock initialized: now 1790506328413150 us; error 31 us; skew 500 ppm
Sep 27 10:52:08 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:08.415846 26045 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:52:08.416685 26045 webserver.cc:460] Webserver started at http://127.9.39.130:44273/ using document root <none> and password file <none>
I20260927 10:52:08.417218 26045 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:08.417331 26045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:08.421005 26045 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:08.422959 26062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.424203 26045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:52:08.424355 26045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "8299a7dcbc694313a24e18cda0a14eff"
format_stamp: "Formatted at 2026-09-27 10:51:25 on dist-test-slave-p9q6"
server_key: "f4edc260a3b7d9179eb91dce3c1ea9ce"
server_key_iv: "da56b6380c1a1d19f29953349f5b0365"
server_key_version: "encryptionkey@0"
I20260927 10:52:08.424888 26045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:08.456398 26045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:08.459296 26045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:08.459494 26045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:08.460085 26045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:08.460896 26045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:08.460966 26045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.461040 26045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:08.461084 26045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.470006 26045 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:43397
I20260927 10:52:08.470033 26175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:43397 every 8 connection(s)
I20260927 10:52:08.471045 26045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:52:08.471786 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26045
I20260927 10:52:08.472337 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:38441
--local_ip_for_outbound_sockets=127.9.39.131
--tserver_master_addrs=127.9.39.190:45913
--webserver_port=34527
--webserver_interface=127.9.39.131
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:52:08.475634 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.473416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:58135)
0927 10:52:08.473995 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.474710 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":135,"threads_started":1}
W20260927 10:52:08.476234 26176 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:45913 (0 consecutive failures): Network error: Failed to ping master at 127.9.39.190:45913: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
I20260927 10:52:08.476878 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.476646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:52579)
0927 10:52:08.476759 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.476828 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 10:52:08.477569 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.477371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:34733)
0927 10:52:08.477471 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.477516 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260927 10:52:08.477792 26176 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 10:52:08.584592 26178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:08.584870 26178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:08.584914 26178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:08.588954 26178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:08.589017 26178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:08.589077 26178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:08.594161 26178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:38441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=34527
--tserver_master_addrs=127.9.39.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:08.595157 26178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:08.596130 26178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:08.605749 26185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:08.605871 26178 server_base.cc:1034] running on GCE node
W20260927 10:52:08.605769 26187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:08.605815 26184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:08.606333 26178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:08.607007 26178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:08.608186 26178 hybrid_clock.cc:648] HybridClock initialized: now 1790506328608163 us; error 42 us; skew 500 ppm
Sep 27 10:52:08 dist-test-slave-p9q6 krb5kdc[24722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:08.610832 26178 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:52:08.611634 26178 webserver.cc:460] Webserver started at http://127.9.39.131:34527/ using document root <none> and password file <none>
I20260927 10:52:08.612187 26178 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:08.612277 26178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:08.615932 26178 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:08.617996 26194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.619261 26178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:08.619395 26178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "9f4a47c18a8440a897610dc3a83c26fe"
format_stamp: "Formatted at 2026-09-27 10:51:26 on dist-test-slave-p9q6"
server_key: "f7f901aee4596290939c2e199ed4e767"
server_key_iv: "2615de49d8b263edd991cb9111e3127b"
server_key_version: "encryptionkey@0"
I20260927 10:52:08.619935 26178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:08.633910 26178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:08.636739 26178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:08.636942 26178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:08.637494 26178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:08.638260 26178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:08.638336 26178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.638401 26178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:08.638453 26178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:08.647642 26178 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:38441
I20260927 10:52:08.647706 26307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:38441 every 8 connection(s)
I20260927 10:52:08.648622 26178 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:08.649801 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26178
I20260927 10:52:08.652386 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.650936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:57981)
0927 10:52:08.651592 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.651688 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":108,"threads_started":1}
W20260927 10:52:08.652863 26308 heartbeater.cc:644] Failed to heartbeat to 127.9.39.190:45913 (0 consecutive failures): Network error: Failed to ping master at 127.9.39.190:45913: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
I20260927 10:52:08.653373 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.653179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:51147)
0927 10:52:08.653279 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.653321 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 10:52:08.653930 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:08.653758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:34101)
0927 10:52:08.653847 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:08.653885 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 10:52:08.654110 26308 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 10:52:09.292161 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:09.291748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:53199)
0927 10:52:09.291993 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:09.292049 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
I20260927 10:52:09.478796 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:09.478355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:37709)
0927 10:52:09.478581 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:09.478633 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
I20260927 10:52:09.655031 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:09.654635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:49297)
0927 10:52:09.654889 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:09.654940 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":106,"threads_started":1}
I20260927 10:52:10.046013 26315 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:10.046197 26315 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:52:10.047376 26315 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
Sep 27 10:52:10 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.131@KRBTEST.COM
Sep 27 10:52:10 dist-test-slave-p9q6 krb5kdc[24722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.130@KRBTEST.COM
I20260927 10:52:10.063275 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.047575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:38441 (local address 127.9.39.129:46065)
0927 10:52:10.047838 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:10.047853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:10.047956 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:10.049521 (+ 1565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:10.049527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:10.050054 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:10.050559 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:10.050578 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.052424 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.052427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:10.054080 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:10.054091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.055152 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.055156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:10.055409 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:10.056180 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:10.056233 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:10.058206 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:10.060565 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:10.060571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:10.060585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:10.060842 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:10.061255 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:10.061258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:10.061260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:10.061424 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:10.062229 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:10.062236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:10.062479 (+ 243us) client_negotiation.cc:769] Sending connection context
0927 10:52:10.062680 (+ 201us) client_negotiation.cc:240] Negotiation successful
0927 10:52:10.063081 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":238,"spinlock_wait_cycles":6912,"thread_start_us":105,"threads_started":1}
I20260927 10:52:10.063376 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.047745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:46065 (local address 127.9.39.131:38441)
0927 10:52:10.048156 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:10.048178 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:10.048213 (+ 35us) server_negotiation.cc:407] Connection header received
0927 10:52:10.048859 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:10.048864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:10.049181 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:10.049477 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:10.050797 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.052264 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.054231 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.055000 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.055177 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:10.058431 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:10.058436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:10.058448 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:10.058474 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:10.060414 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:10.060962 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:10.060968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:10.060973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:10.061130 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:10.061527 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:10.061530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:10.061532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:10.062087 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:10.062206 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:10.062892 (+ 686us) server_negotiation.cc:299] Negotiation successful
0927 10:52:10.063191 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":63,"threads_started":1}
I20260927 10:52:10.063398 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.047594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43397 (local address 127.9.39.129:55689)
0927 10:52:10.047932 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:10.047945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:10.048048 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:10.049521 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:10.049527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:10.050055 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:10.050556 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:10.050572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.052424 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.052427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:10.054080 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:10.054091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.055152 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.055156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:10.055411 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:10.056244 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:10.056256 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:10.058344 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:10.060750 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:10.060755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:10.060758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:10.060973 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:10.061717 (+ 744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:10.061720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:10.061722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:10.061819 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:10.062571 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:10.062574 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:10.062694 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 10:52:10.062843 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 10:52:10.063182 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":477,"spinlock_wait_cycles":26496,"thread_start_us":62,"threads_started":1}
I20260927 10:52:10.064002 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.047930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:55689 (local address 127.9.39.130:43397)
0927 10:52:10.048264 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:10.048274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:10.048304 (+ 30us) server_negotiation.cc:407] Connection header received
0927 10:52:10.048895 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:10.048902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:10.049183 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:10.049478 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:10.050747 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.052264 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.054231 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:10.055000 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:10.055188 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:10.058533 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:10.058537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:10.058548 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:10.058576 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:10.060570 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:10.061115 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:10.061121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:10.061125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:10.061249 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:10.061934 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:10.061937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:10.061938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:10.062411 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:10.062534 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:10.062980 (+ 446us) server_negotiation.cc:299] Negotiation successful
0927 10:52:10.063255 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":97,"threads_started":1}
W20260927 10:52:10.066291 25930 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
W20260927 10:52:10.066448 25931 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
I20260927 10:52:10.066522 25931 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01; no voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe
I20260927 10:52:10.066774 26315 raft_consensus.cc:2726] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:52:10.293097 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.292838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:48425)
0927 10:52:10.292962 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:10.293015 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 10:52:10.479912 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.479476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:38509)
0927 10:52:10.479729 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:10.479787 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":70,"threads_started":1}
I20260927 10:52:10.656188 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:10.655729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:48759)
0927 10:52:10.655990 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:10.656049 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260927 10:52:11.294267 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:11.293822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:43089)
0927 10:52:11.294115 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:11.294171 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":67,"threads_started":1}
I20260927 10:52:11.480942 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:11.480594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:59535)
0927 10:52:11.480809 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:11.480858 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260927 10:52:11.657156 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:11.656795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:51841)
0927 10:52:11.657014 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:11.657065 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":61,"threads_started":1}
I20260927 10:52:11.754807 26325 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:11.754940 26325 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:52:11.755437 26325 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
W20260927 10:52:11.756313 25930 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
W20260927 10:52:11.756404 25931 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
I20260927 10:52:11.756466 25931 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01; no voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe
I20260927 10:52:11.756618 26325 raft_consensus.cc:2726] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:52:12.295424 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:12.294998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:36323)
0927 10:52:12.295267 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:12.295330 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
I20260927 10:52:12.481986 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:12.481605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:44487)
0927 10:52:12.481849 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:12.481900 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260927 10:52:12.658427 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:12.658031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:58123)
0927 10:52:12.658287 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:12.658342 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260927 10:52:13.296893 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:13.296444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:43897)
0927 10:52:13.296723 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:13.296795 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260927 10:52:13.483397 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:13.482941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:44239)
0927 10:52:13.483234 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:13.483300 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
I20260927 10:52:13.659626 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:13.659203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:33009)
0927 10:52:13.659473 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:13.659534 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
I20260927 10:52:14.298084 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:14.297697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:42603)
0927 10:52:14.297942 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:14.297991 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260927 10:52:14.464396 26333 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:14.464522 26333 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:52:14.464906 26333 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
W20260927 10:52:14.465684 25930 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
W20260927 10:52:14.465834 25931 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
I20260927 10:52:14.465934 25931 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01; no voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe
I20260927 10:52:14.466111 26333 raft_consensus.cc:2726] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:52:14.484589 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:14.484168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:52237)
0927 10:52:14.484447 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:14.484498 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":109,"threads_started":1}
I20260927 10:52:14.660810 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:14.660413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:55463)
0927 10:52:14.660667 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:14.660720 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
I20260927 10:52:15.300069 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:15.299374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:46131)
0927 10:52:15.299826 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:15.299906 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":126,"threads_started":1}
I20260927 10:52:15.485975 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:15.485450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:34571)
0927 10:52:15.485777 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:15.485842 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":75,"threads_started":1}
I20260927 10:52:15.661967 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:15.661528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:59661)
0927 10:52:15.661826 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:15.661877 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":114,"threads_started":1}
I20260927 10:52:16.301374 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:16.300919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:45375)
0927 10:52:16.301201 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:16.301275 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260927 10:52:16.487407 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:16.486928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:44245)
0927 10:52:16.487223 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:16.487289 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260927 10:52:16.663285 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:16.662852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:36731)
0927 10:52:16.663119 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:16.663179 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260927 10:52:17.302685 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:17.302256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:33643)
0927 10:52:17.302533 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:17.302590 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
I20260927 10:52:17.488745 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:17.488234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:43827)
0927 10:52:17.488528 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:17.488596 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":76,"threads_started":1}
I20260927 10:52:17.664557 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:17.664028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:50533)
0927 10:52:17.664301 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:17.664402 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
I20260927 10:52:17.917359 26345 raft_consensus.cc:482] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:17.917544 26345 raft_consensus.cc:504] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4a47c18a8440a897610dc3a83c26fe" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 38441 } } peers { permanent_uuid: "8299a7dcbc694313a24e18cda0a14eff" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43397 } } peers { permanent_uuid: "8809ed3742434326882b54f8a04a0e01" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 36551 } }
I20260927 10:52:17.918071 26345 leader_election.cc:288] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441), 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397)
W20260927 10:52:17.918936 25930 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f4a47c18a8440a897610dc3a83c26fe (127.9.39.131:38441): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
W20260927 10:52:17.919025 25931 leader_election.cc:341] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8299a7dcbc694313a24e18cda0a14eff (127.9.39.130:43397): Not found: Tablet not found: 4db40513720a42cca682226fb56d68ac
I20260927 10:52:17.919080 25930 leader_election.cc:302] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8809ed3742434326882b54f8a04a0e01; no voters: 8299a7dcbc694313a24e18cda0a14eff, 9f4a47c18a8440a897610dc3a83c26fe
I20260927 10:52:17.919235 26345 raft_consensus.cc:2726] T 4db40513720a42cca682226fb56d68ac P 8809ed3742434326882b54f8a04a0e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 10:52:18.303985 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:18.303613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:47851)
0927 10:52:18.303842 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:18.303891 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
I20260927 10:52:18.489869 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:18.489480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:42779)
0927 10:52:18.489709 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:18.489763 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":70,"threads_started":1}
I20260927 10:52:18.665753 26348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:18.665350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:57615)
0927 10:52:18.665604 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:18.665659 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":70,"threads_started":1}
I20260927 10:52:19.305115 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:19.304691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:55989)
0927 10:52:19.304959 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:19.305018 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":80,"threads_started":1}
I20260927 10:52:19.491120 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:19.490663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:38037)
0927 10:52:19.490961 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:19.491023 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
I20260927 10:52:19.667057 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:19.666527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:51983)
0927 10:52:19.666854 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:19.666926 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":81,"threads_started":1}
I20260927 10:52:20.306255 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:20.305842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:56601)
0927 10:52:20.306108 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:20.306161 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":88,"threads_started":1}
I20260927 10:52:20.492264 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:20.491866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:58807)
0927 10:52:20.492108 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:20.492166 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260927 10:52:20.668234 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:20.667815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:43483)
0927 10:52:20.668072 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:20.668132 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260927 10:52:21.307610 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:21.307164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:52461)
0927 10:52:21.307446 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:21.307510 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260927 10:52:21.493738 26356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:21.493254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:56395)
0927 10:52:21.493549 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:21.493615 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":98,"threads_started":1}
I20260927 10:52:21.669446 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:21.668991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:45407)
0927 10:52:21.669250 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:21.669319 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":65,"threads_started":1}
I20260927 10:52:22.308746 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:22.308316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:46773)
0927 10:52:22.308593 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:22.308646 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":113,"threads_started":1}
I20260927 10:52:22.494971 26359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:22.494485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:51297)
0927 10:52:22.494815 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:22.494877 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":79,"threads_started":1}
I20260927 10:52:22.670542 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:22.670167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:45325)
0927 10:52:22.670393 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:22.670449 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260927 10:52:23.310146 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:23.309698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.129:46013)
0927 10:52:23.309986 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:23.310048 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":88,"threads_started":1}
I20260927 10:52:23.496428 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:23.495923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.130:38835)
0927 10:52:23.496214 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:23.496280 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":75,"threads_started":1}
I20260927 10:52:23.660645 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 25872
I20260927 10:52:23.666325 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26045
I20260927 10:52:23.671763 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26178
I20260927 10:52:23.672101 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:23.671527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:45913 (local address 127.9.39.131:46165)
0927 10:52:23.671904 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:23.671971 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.190:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":196,"threads_started":1}
2026-09-27T10:52:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59637 ms)
[----------] 30 tests from SecurityITest (238763 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:23 dist-test-slave-p9q6 krb5kdc[26366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:23 dist-test-slave-p9q6 krb5kdc[26366](info): set up 2 sockets
Sep 27 10:52:23 dist-test-slave-p9q6 krb5kdc[26366](info): commencing operation
krb5kdc: starting...
W20260927 10:52:24.706449 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:24 dist-test-slave-p9q6 krb5kdc[26366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:24.829535 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:34351
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:34351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:24.937516 26379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:24.937829 26379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:24.937897 26379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:24.941915 26379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:24.941998 26379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:24.942036 26379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:24.942091 26379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:24.942124 26379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:24.947355 26379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:34351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:34351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:24.948375 26379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:24.949399 26379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:24.958506 26384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:24.958564 26379 server_base.cc:1034] running on GCE node
W20260927 10:52:24.958513 26387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:24.958536 26385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:24.959368 26379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:24.960479 26379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:24.961683 26379 hybrid_clock.cc:648] HybridClock initialized: now 1790506344961660 us; error 43 us; skew 500 ppm
Sep 27 10:52:24 dist-test-slave-p9q6 krb5kdc[26366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506344, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:24.964548 26379 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:24.965490 26379 webserver.cc:460] Webserver started at http://127.9.39.190:45977/ using document root <none> and password file <none>
I20260927 10:52:24.966041 26379 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:24.966137 26379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:24.966363 26379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:24.968462 26379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "83edd443af7a4ca2871f9cbb386fb903"
format_stamp: "Formatted at 2026-09-27 10:52:24 on dist-test-slave-p9q6"
server_key: "b5548866f964a2ee48c73893086d097c"
server_key_iv: "77150168dd2b8ab38172c92f63304f40"
server_key_version: "encryptionkey@0"
I20260927 10:52:24.969025 26379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "83edd443af7a4ca2871f9cbb386fb903"
format_stamp: "Formatted at 2026-09-27 10:52:24 on dist-test-slave-p9q6"
server_key: "b5548866f964a2ee48c73893086d097c"
server_key_iv: "77150168dd2b8ab38172c92f63304f40"
server_key_version: "encryptionkey@0"
I20260927 10:52:24.972743 26379 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:52:24.975217 26394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:24.976419 26379 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:24.976552 26379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "83edd443af7a4ca2871f9cbb386fb903"
format_stamp: "Formatted at 2026-09-27 10:52:24 on dist-test-slave-p9q6"
server_key: "b5548866f964a2ee48c73893086d097c"
server_key_iv: "77150168dd2b8ab38172c92f63304f40"
server_key_version: "encryptionkey@0"
I20260927 10:52:24.976660 26379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:24.998086 26379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:25.001144 26379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:25.001346 26379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:25.008518 26379 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:34351
I20260927 10:52:25.008543 26446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:34351 every 8 connection(s)
I20260927 10:52:25.009465 26379 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:25.012372 26447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:25.015349 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26379
I20260927 10:52:25.015498 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:25.015808 9374 external_mini_cluster.cc:1428] Setting key 9f7ea24cd34e88c462ed12b922472356
I20260927 10:52:25.018430 26447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: Bootstrap starting.
I20260927 10:52:25.020581 26447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:25.021309 26447 log.cc:824] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:25.023379 26447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: No bootstrap required, opened a new log
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:25.027460 26447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83edd443af7a4ca2871f9cbb386fb903" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34351 } }
I20260927 10:52:25.027725 26447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:25.027786 26447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83edd443af7a4ca2871f9cbb386fb903, State: Initialized, Role: FOLLOWER
I20260927 10:52:25.028220 26447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83edd443af7a4ca2871f9cbb386fb903" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34351 } }
I20260927 10:52:25.028360 26447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:25.028425 26447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:25.028517 26447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:25.029389 26447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83edd443af7a4ca2871f9cbb386fb903" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34351 } }
I20260927 10:52:25.029755 26447 leader_election.cc:302] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83edd443af7a4ca2871f9cbb386fb903; no voters:
I20260927 10:52:25.030037 26447 leader_election.cc:288] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:25.030133 26452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:25.030320 26452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [term 1 LEADER]: Becoming Leader. State: Replica: 83edd443af7a4ca2871f9cbb386fb903, State: Running, Role: LEADER
I20260927 10:52:25.030529 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.017337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.9.39.190:34351)
0927 10:52:25.017778 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.017795 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.017836 (+ 41us) server_negotiation.cc:407] Connection header received
0927 10:52:25.018466 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.018482 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.018884 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.019225 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:25.019966 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.020982 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.021862 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.022560 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.022743 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.025247 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:25.025253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:25.025270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:25.025300 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:25.027401 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.027878 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.027885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.027890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.028008 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.028292 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.028295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.028297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.028794 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:25.029005 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.029262 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.029591 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":127,"threads_started":1}
I20260927 10:52:25.031317 26447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:25.031677 26452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83edd443af7a4ca2871f9cbb386fb903" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34351 } }
I20260927 10:52:25.032886 26453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83edd443af7a4ca2871f9cbb386fb903. Latest consensus state: current_term: 1 leader_uuid: "83edd443af7a4ca2871f9cbb386fb903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83edd443af7a4ca2871f9cbb386fb903" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34351 } } }
I20260927 10:52:25.032992 26453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:25.033607 26454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83edd443af7a4ca2871f9cbb386fb903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83edd443af7a4ca2871f9cbb386fb903" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34351 } } }
I20260927 10:52:25.033691 26454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903 [sys.catalog]: This master's current role is: LEADER
W20260927 10:52:25.035851 26467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:52:25.035913 26467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:52:25.035984 26466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:25.036787 26466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:25.041462 26466 catalog_manager.cc:1347] Generated new cluster ID: 66e7e1c5ba464c08b837d92845069281
I20260927 10:52:25.041522 26466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:25.064361 26466 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:25.065289 26466 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:25.076529 26466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: Generated new TSK 0
I20260927 10:52:25.077298 26466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:52:25.109608 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34351
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:25.219553 26473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:25.219815 26473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:25.219859 26473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:25.223886 26473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:25.223937 26473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:25.223992 26473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:25.229050 26473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:25.230002 26473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:25.230978 26473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:25.240406 26478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:25.240451 26473 server_base.cc:1034] running on GCE node
W20260927 10:52:25.240415 26479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:25.240651 26481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:25.241020 26473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:25.241628 26473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:25.242776 26473 hybrid_clock.cc:648] HybridClock initialized: now 1790506345242757 us; error 36 us; skew 500 ppm
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506345, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:25.245417 26473 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:52:25.246275 26473 webserver.cc:460] Webserver started at http://127.9.39.129:36585/ using document root <none> and password file <none>
I20260927 10:52:25.246870 26473 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:25.246953 26473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:25.247167 26473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:25.249166 26473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "d3429c24919e40eaa098fb6d75923324"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "897f09d830ab8e85138a4af59ffb9ac6"
server_key_iv: "ec2286be96eac6350539f0d5056b8118"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.249725 26473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "d3429c24919e40eaa098fb6d75923324"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "897f09d830ab8e85138a4af59ffb9ac6"
server_key_iv: "ec2286be96eac6350539f0d5056b8118"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.253623 26473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260927 10:52:25.256171 26488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.257508 26473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:25.257651 26473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "d3429c24919e40eaa098fb6d75923324"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "897f09d830ab8e85138a4af59ffb9ac6"
server_key_iv: "ec2286be96eac6350539f0d5056b8118"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.257771 26473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:25.273996 26473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:25.277338 26473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:25.277561 26473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:25.278182 26473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:25.279093 26473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:25.279161 26473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.279227 26473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:25.279265 26473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.288786 26473 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:42185
I20260927 10:52:25.288801 26601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:42185 every 8 connection(s)
I20260927 10:52:25.289728 26473 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:25.295630 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26473
I20260927 10:52:25.295797 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:52:25.296096 9374 external_mini_cluster.cc:1428] Setting key a35523f21a81a4af39a060dfb5d1b0ec
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506345, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:25.304873 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.291780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:47187 (local address 127.9.39.190:34351)
0927 10:52:25.291936 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.291939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.292826 (+ 887us) server_negotiation.cc:407] Connection header received
0927 10:52:25.294252 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.294255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.294314 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.294399 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:25.295893 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.296501 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.297986 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.298416 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.298545 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.301209 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:25.301213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:25.301216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:25.301245 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:25.302774 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.303287 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.303291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.303293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.303372 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.303716 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.303719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.303721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.303911 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:25.304017 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.304555 (+ 538us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.304697 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:52:25.305773 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.292161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:34351 (local address 127.9.39.129:47187)
0927 10:52:25.292660 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.292706 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.294034 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.294597 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.294605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.295031 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:25.295664 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.295687 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.296633 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.296637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.297853 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.297863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.298587 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.298591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.298795 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.299454 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:25.299471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:25.301042 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:25.302885 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:25.302891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:25.302908 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:25.303179 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:25.303466 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:25.303469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:25.303471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:25.303616 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:25.304015 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:25.304022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:25.304305 (+ 283us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.304505 (+ 200us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.304853 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":133,"threads_started":1}
I20260927 10:52:25.307286 26602 heartbeater.cc:347] Connected to a master server at 127.9.39.190:34351
I20260927 10:52:25.307638 26602 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
I20260927 10:52:25.308240 26602 heartbeater.cc:511] Master 127.9.39.190:34351 requested a full tablet report, sending...
I20260927 10:52:25.310024 26411 ts_manager.cc:194] Registered new tserver with Master: d3429c24919e40eaa098fb6d75923324 (127.9.39.129:42185)
I20260927 10:52:25.311271 26411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:47187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:52:25.325560 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34351
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:25.439040 26607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:25.439432 26607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:25.439538 26607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:25.445065 26607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:25.445164 26607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:25.445281 26607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:25.450914 26607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:25.452024 26607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:25.453063 26607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:25.462942 26613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:25.462936 26612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:25.462942 26615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:25.463342 26607 server_base.cc:1034] running on GCE node
I20260927 10:52:25.463797 26607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:25.464462 26607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:25.465656 26607 hybrid_clock.cc:648] HybridClock initialized: now 1790506345465634 us; error 40 us; skew 500 ppm
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506345, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:25.468443 26607 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:52:25.469376 26607 webserver.cc:460] Webserver started at http://127.9.39.130:42707/ using document root <none> and password file <none>
I20260927 10:52:25.469893 26607 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:25.469954 26607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:25.470127 26607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:25.472249 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "a1c5527276094ac3a10d111fc1d8b1e3"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "f96c2dddcb400258514302763117107d"
server_key_iv: "f326c2695a0f7ad6ad407980a42719d1"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.472774 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "a1c5527276094ac3a10d111fc1d8b1e3"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "f96c2dddcb400258514302763117107d"
server_key_iv: "f326c2695a0f7ad6ad407980a42719d1"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.476665 26607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 10:52:25.479182 26622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.480454 26607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:25.480561 26607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "a1c5527276094ac3a10d111fc1d8b1e3"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "f96c2dddcb400258514302763117107d"
server_key_iv: "f326c2695a0f7ad6ad407980a42719d1"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.480659 26607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:25.497529 26607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:25.500484 26607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:25.500636 26607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:25.501158 26607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:25.501890 26607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:25.501936 26607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.501974 26607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:25.501996 26607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.510934 26607 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:34997
I20260927 10:52:25.510952 26735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:34997 every 8 connection(s)
I20260927 10:52:25.511809 26607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:52:25.512182 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26607
I20260927 10:52:25.512332 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:52:25.512691 9374 external_mini_cluster.cc:1428] Setting key d34607f7e16a28727b69285c1b3d3a57
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506345, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:25.530781 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.513527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:34763 (local address 127.9.39.190:34351)
0927 10:52:25.513656 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.513660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.514621 (+ 961us) server_negotiation.cc:407] Connection header received
0927 10:52:25.516461 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.516465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.516515 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.516620 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:25.518062 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.518684 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.520571 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.520792 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.520870 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.526715 (+ 5845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:25.526722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:25.526728 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:25.526759 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:25.528349 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.528890 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.528897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.528902 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.529012 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.529347 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.529354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.529359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.529654 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:25.529786 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.530332 (+ 546us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.530496 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 10:52:25.531527 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.513920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:34351 (local address 127.9.39.130:34763)
0927 10:52:25.514449 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.514506 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.516262 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.516761 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.516768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.517220 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:25.517905 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.517927 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.518845 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.518849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.520441 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.520454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.520939 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.520946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.521173 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.521817 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:25.521842 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:25.526478 (+ 4636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:25.528491 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:25.528496 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:25.528508 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:25.528784 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:25.529120 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:25.529122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:25.529123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:25.529248 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:25.529789 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:25.529798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:25.530056 (+ 258us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.530239 (+ 183us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.530504 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":152,"threads_started":1}
I20260927 10:52:25.532842 26736 heartbeater.cc:347] Connected to a master server at 127.9.39.190:34351
I20260927 10:52:25.533074 26736 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:25.533655 26736 heartbeater.cc:511] Master 127.9.39.190:34351 requested a full tablet report, sending...
I20260927 10:52:25.534782 26411 ts_manager.cc:194] Registered new tserver with Master: a1c5527276094ac3a10d111fc1d8b1e3 (127.9.39.130:34997)
I20260927 10:52:25.535449 26411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:34763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:52:25.541582 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34351
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:25.649418 26741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:25.649657 26741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:25.649698 26741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:25.653551 26741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:25.653599 26741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:25.653656 26741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:25.658823 26741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:34351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:25.659838 26741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:25.660805 26741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:52:25.670308 26741 server_base.cc:1034] running on GCE node
W20260927 10:52:25.670255 26746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:25.670259 26749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:25.670239 26747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:25.670862 26741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:25.671444 26741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:25.672592 26741 hybrid_clock.cc:648] HybridClock initialized: now 1790506345672576 us; error 26 us; skew 500 ppm
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506345, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:25.675194 26741 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:52:25.676062 26741 webserver.cc:460] Webserver started at http://127.9.39.131:38015/ using document root <none> and password file <none>
I20260927 10:52:25.676608 26741 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:25.676694 26741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:25.676913 26741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:25.678934 26741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "e008146610ce442fa16997c70196f130"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "8c4106be3c3ca1e3d684ce7bb70a7e80"
server_key_iv: "80adc7be33318ba800ec06dacd1c2351"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.679479 26741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "e008146610ce442fa16997c70196f130"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "8c4106be3c3ca1e3d684ce7bb70a7e80"
server_key_iv: "80adc7be33318ba800ec06dacd1c2351"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.683072 26741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:52:25.685475 26756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.686676 26741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:25.686856 26741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "e008146610ce442fa16997c70196f130"
format_stamp: "Formatted at 2026-09-27 10:52:25 on dist-test-slave-p9q6"
server_key: "8c4106be3c3ca1e3d684ce7bb70a7e80"
server_key_iv: "80adc7be33318ba800ec06dacd1c2351"
server_key_version: "encryptionkey@0"
I20260927 10:52:25.686962 26741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:25.700946 26741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:25.703707 26741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:25.703899 26741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:25.704476 26741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:25.705255 26741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:25.705319 26741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.705385 26741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:25.705430 26741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:25.714437 26741 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:42807
I20260927 10:52:25.714463 26869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:42807 every 8 connection(s)
I20260927 10:52:25.715343 26741 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:25.717393 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26741
I20260927 10:52:25.717520 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:52:25.717790 9374 external_mini_cluster.cc:1428] Setting key a66b2c9416168bc9fcaee4519d2054aa
Sep 27 10:52:25 dist-test-slave-p9q6 krb5kdc[26366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506345, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:25.728269 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.717132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:50021 (local address 127.9.39.190:34351)
0927 10:52:25.717306 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.717311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.718189 (+ 878us) server_negotiation.cc:407] Connection header received
0927 10:52:25.719521 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.719524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.719577 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.719660 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:25.720730 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.721251 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.722381 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.722583 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.722658 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.725098 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:25.725101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:25.725103 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:25.725125 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:25.726524 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.727021 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.727023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.727025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.727102 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.727377 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.727379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.727380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.727537 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:25.727611 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.728054 (+ 443us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.728155 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:52:25.728996 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.717544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:34351 (local address 127.9.39.131:50021)
0927 10:52:25.718032 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.718082 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.719340 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.719835 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.719841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.720170 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:25.720616 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.720627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.721375 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.721378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.722273 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.722283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.722801 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.722806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.722956 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.723507 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:25.723523 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:25.724953 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:25.726639 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:25.726644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:25.726654 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:25.726942 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:25.727174 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:25.727176 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:25.727177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:25.727300 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:25.727640 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:25.727645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:25.727862 (+ 217us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.728001 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.728228 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":128,"threads_started":1}
I20260927 10:52:25.730073 26870 heartbeater.cc:347] Connected to a master server at 127.9.39.190:34351
I20260927 10:52:25.730337 26870 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:25.730957 26870 heartbeater.cc:511] Master 127.9.39.190:34351 requested a full tablet report, sending...
I20260927 10:52:25.732122 26411 ts_manager.cc:194] Registered new tserver with Master: e008146610ce442fa16997c70196f130 (127.9.39.131:42807)
I20260927 10:52:25.732787 26411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:50021
I20260927 10:52:25.743219 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:52:25.753752 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.745372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.9.39.190:34351)
0927 10:52:25.745514 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.745520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.745753 (+ 233us) server_negotiation.cc:407] Connection header received
0927 10:52:25.745916 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.745920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.745970 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.746071 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:25.746879 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.747408 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.748329 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.748557 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.748695 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.750389 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:25.750394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:25.750403 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:25.750426 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:25.751941 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.752424 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.752430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.752432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.752529 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:25.752822 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:25.752827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:25.752829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:25.753067 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:25.753212 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.753457 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.753574 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 10:52:25.756553 26411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:52:25.759696 26411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:52:25.773686 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.768596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.9.39.129:42185)
0927 10:52:25.769315 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.769328 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.769347 (+ 19us) server_negotiation.cc:407] Connection header received
0927 10:52:25.769475 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.769479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.769643 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.769775 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:25.770042 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.770826 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.772210 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.773030 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.773141 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.773288 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.773447 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.773536 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":116,"threads_started":1}
I20260927 10:52:25.773679 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.767996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:42185 (local address 127.0.0.1:45576)
0927 10:52:25.768640 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.768691 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.768822 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.769794 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.769798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.769816 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:25.769935 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.769941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.770948 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.770951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.772067 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.772077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.773141 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.773145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.773245 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.773279 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.773414 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.773529 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":3,"spinlock_wait_cycles":4736,"thread_start_us":111,"threads_started":1}
I20260927 10:52:25.774461 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.768048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:42807 (local address 127.0.0.1:36342)
0927 10:52:25.769000 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.769012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.769083 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.769828 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.769834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.769859 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:25.769958 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.769965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.771496 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.771499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.772652 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.772662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.774003 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.774007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.774115 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.774140 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.774227 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.774313 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":59,"thread_start_us":86,"threads_started":1}
I20260927 10:52:25.774478 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.768771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.9.39.131:42807)
0927 10:52:25.769161 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.769175 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.769201 (+ 26us) server_negotiation.cc:407] Connection header received
0927 10:52:25.769284 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.769293 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.769500 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.769677 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:25.770090 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.771348 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.772777 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.773648 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.773767 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.773939 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.774264 (+ 325us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.774349 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":99,"threads_started":1}
I20260927 10:52:25.775113 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.767802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:34997 (local address 127.0.0.1:46198)
0927 10:52:25.768640 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.768691 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.768822 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.769613 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.769622 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.769650 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:25.769785 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.769797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.771572 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.771576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.772849 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.772861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.774372 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.774376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.774464 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.774497 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.774891 (+ 394us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.774975 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":45,"spinlock_wait_cycles":6528,"thread_start_us":74,"threads_started":1}
I20260927 10:52:25.775688 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.767931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.9.39.130:34997)
0927 10:52:25.769108 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.769124 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.769149 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:52:25.769227 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.769232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.769442 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.769606 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:25.769930 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.771302 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.772979 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.774240 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.775299 (+ 1059us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.775455 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.775496 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.775573 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":158,"threads_started":1}
I20260927 10:52:25.776789 26536 tablet_service.cc:1467] Processing CreateTablet for tablet dcbcb4a91ce44549aa03b2dcaa704bce (DEFAULT_TABLE table=test-table [id=bab2d9bb5f214b0fa3199dd993a65f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:25.777700 26536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbcb4a91ce44549aa03b2dcaa704bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:25.778330 26804 tablet_service.cc:1467] Processing CreateTablet for tablet dcbcb4a91ce44549aa03b2dcaa704bce (DEFAULT_TABLE table=test-table [id=bab2d9bb5f214b0fa3199dd993a65f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:25.778330 26670 tablet_service.cc:1467] Processing CreateTablet for tablet dcbcb4a91ce44549aa03b2dcaa704bce (DEFAULT_TABLE table=test-table [id=bab2d9bb5f214b0fa3199dd993a65f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:25.779386 26670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbcb4a91ce44549aa03b2dcaa704bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:25.779392 26804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbcb4a91ce44549aa03b2dcaa704bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:25.784247 26887 tablet_bootstrap.cc:492] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324: Bootstrap starting.
I20260927 10:52:25.786202 26887 tablet_bootstrap.cc:654] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:25.787156 26888 tablet_bootstrap.cc:492] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Bootstrap starting.
I20260927 10:52:25.787420 26887 log.cc:824] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:25.789166 26887 tablet_bootstrap.cc:492] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324: No bootstrap required, opened a new log
I20260927 10:52:25.789323 26887 ts_tablet_manager.cc:1397] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:52:25.789321 26888 tablet_bootstrap.cc:654] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:25.790060 26888 log.cc:824] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:25.790753 26889 tablet_bootstrap.cc:492] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Bootstrap starting.
I20260927 10:52:25.792105 26888 tablet_bootstrap.cc:492] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: No bootstrap required, opened a new log
I20260927 10:52:25.792289 26888 ts_tablet_manager.cc:1397] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:52:25.792603 26889 tablet_bootstrap.cc:654] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:25.793170 26887 raft_consensus.cc:348] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.793521 26887 raft_consensus.cc:374] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:25.793598 26887 raft_consensus.cc:729] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3429c24919e40eaa098fb6d75923324, State: Initialized, Role: FOLLOWER
I20260927 10:52:25.793613 26889 log.cc:824] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:25.794071 26887 consensus_queue.cc:260] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.794800 26602 heartbeater.cc:503] Master 127.9.39.190:34351 was elected leader, sending a full tablet report...
I20260927 10:52:25.795136 26887 ts_tablet_manager.cc:1428] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 10:52:25.796133 26888 raft_consensus.cc:348] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.796386 26888 raft_consensus.cc:374] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:25.796437 26888 raft_consensus.cc:729] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c5527276094ac3a10d111fc1d8b1e3, State: Initialized, Role: FOLLOWER
I20260927 10:52:25.796775 26889 tablet_bootstrap.cc:492] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: No bootstrap required, opened a new log
I20260927 10:52:25.796831 26888 consensus_queue.cc:260] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.796963 26889 ts_tablet_manager.cc:1397] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:52:25.797477 26736 heartbeater.cc:503] Master 127.9.39.190:34351 was elected leader, sending a full tablet report...
I20260927 10:52:25.797827 26888 ts_tablet_manager.cc:1428] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260927 10:52:25.797829 26893 raft_consensus.cc:482] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:25.797956 26893 raft_consensus.cc:504] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.799014 26893 leader_election.cc:288] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1c5527276094ac3a10d111fc1d8b1e3 (127.9.39.130:34997), e008146610ce442fa16997c70196f130 (127.9.39.131:42807)
I20260927 10:52:25.800710 26889 raft_consensus.cc:348] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.800957 26889 raft_consensus.cc:374] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:25.801010 26889 raft_consensus.cc:729] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008146610ce442fa16997c70196f130, State: Initialized, Role: FOLLOWER
I20260927 10:52:25.801429 26889 consensus_queue.cc:260] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.802284 26889 ts_tablet_manager.cc:1428] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260927 10:52:25.802615 26870 heartbeater.cc:503] Master 127.9.39.190:34351 was elected leader, sending a full tablet report...
I20260927 10:52:25.803435 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.799405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:55069 (local address 127.9.39.130:34997)
0927 10:52:25.799515 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.799520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.799536 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:52:25.799582 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.799596 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.799643 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.799713 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:25.800031 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.800776 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.802242 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.802888 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.803008 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.803058 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.803261 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.803324 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 10:52:25.803439 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.799193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:34997 (local address 127.9.39.129:55069)
0927 10:52:25.799313 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.799330 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.799415 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.799770 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.799773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.799792 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:25.799901 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.799907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.800909 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.800912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.802115 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.802126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.803010 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.803013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.803129 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.803154 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.803239 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.803310 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 10:52:25.803967 26690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbcb4a91ce44549aa03b2dcaa704bce" candidate_uuid: "d3429c24919e40eaa098fb6d75923324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" is_pre_election: true
I20260927 10:52:25.804222 26690 raft_consensus.cc:2454] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3429c24919e40eaa098fb6d75923324 in term 0.
I20260927 10:52:25.804683 26491 leader_election.cc:302] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1c5527276094ac3a10d111fc1d8b1e3, d3429c24919e40eaa098fb6d75923324; no voters:
I20260927 10:52:25.804934 26893 raft_consensus.cc:2781] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:52:25.805008 26893 raft_consensus.cc:482] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:52:25.805086 26893 raft_consensus.cc:3037] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:25.805109 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.799579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:57407 (local address 127.9.39.131:42807)
0927 10:52:25.799714 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.799719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.800999 (+ 1280us) server_negotiation.cc:407] Connection header received
0927 10:52:25.801197 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.801201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.801245 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.801289 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:25.801712 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.802420 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.803850 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.804451 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.804536 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.804580 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.804925 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.804988 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:52:25.805136 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.799264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:42807 (local address 127.9.39.129:57407)
0927 10:52:25.800902 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.800919 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:25.801058 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:25.801434 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:25.801436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:25.801452 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:25.801571 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.801581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.802558 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.802561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.803705 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:25.803715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.804705 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.804712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:25.804814 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.804836 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:52:25.804909 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 10:52:25.804973 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1576,"thread_start_us":83,"threads_started":1}
I20260927 10:52:25.805636 26823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbcb4a91ce44549aa03b2dcaa704bce" candidate_uuid: "d3429c24919e40eaa098fb6d75923324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008146610ce442fa16997c70196f130" is_pre_election: true
I20260927 10:52:25.805881 26823 raft_consensus.cc:2454] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3429c24919e40eaa098fb6d75923324 in term 0.
I20260927 10:52:25.806413 26893 raft_consensus.cc:504] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.806798 26893 leader_election.cc:288] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [CANDIDATE]: Term 1 election: Requested vote from peers a1c5527276094ac3a10d111fc1d8b1e3 (127.9.39.130:34997), e008146610ce442fa16997c70196f130 (127.9.39.131:42807)
I20260927 10:52:25.807128 26690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbcb4a91ce44549aa03b2dcaa704bce" candidate_uuid: "d3429c24919e40eaa098fb6d75923324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c5527276094ac3a10d111fc1d8b1e3"
I20260927 10:52:25.807160 26823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbcb4a91ce44549aa03b2dcaa704bce" candidate_uuid: "d3429c24919e40eaa098fb6d75923324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008146610ce442fa16997c70196f130"
I20260927 10:52:25.807247 26690 raft_consensus.cc:3037] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:25.807261 26823 raft_consensus.cc:3037] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:25.808161 26823 raft_consensus.cc:2454] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3429c24919e40eaa098fb6d75923324 in term 1.
I20260927 10:52:25.808183 26690 raft_consensus.cc:2454] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3429c24919e40eaa098fb6d75923324 in term 1.
I20260927 10:52:25.808456 26492 leader_election.cc:302] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3429c24919e40eaa098fb6d75923324, e008146610ce442fa16997c70196f130; no voters:
I20260927 10:52:25.808632 26893 raft_consensus.cc:2781] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:25.808809 26893 raft_consensus.cc:686] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [term 1 LEADER]: Becoming Leader. State: Replica: d3429c24919e40eaa098fb6d75923324, State: Running, Role: LEADER
I20260927 10:52:25.809115 26893 consensus_queue.cc:237] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } }
I20260927 10:52:25.811868 26410 catalog_manager.cc:5495] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3429c24919e40eaa098fb6d75923324 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "d3429c24919e40eaa098fb6d75923324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3429c24919e40eaa098fb6d75923324" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 42185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 } health_report { overall_health: UNKNOWN } } }
I20260927 10:52:25.822343 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.819019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.9.39.129:42185)
0927 10:52:25.819161 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:25.819184 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:25.819197 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:52:25.819295 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:25.819297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:25.819345 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:25.819444 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:52:25.819672 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.820153 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.821013 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:25.821223 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:25.821316 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:25.821588 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:52:25.821978 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:52:25.822129 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:25.822165 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 10:52:25.822214 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:52:25.827715 26690 raft_consensus.cc:1261] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 1 FOLLOWER]: Refusing update from remote peer d3429c24919e40eaa098fb6d75923324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:25.827721 26823 raft_consensus.cc:1261] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 1 FOLLOWER]: Refusing update from remote peer d3429c24919e40eaa098fb6d75923324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:25.828365 26897 consensus_queue.cc:1035] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1c5527276094ac3a10d111fc1d8b1e3" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:25.828534 26893 consensus_queue.cc:1035] T dcbcb4a91ce44549aa03b2dcaa704bce P d3429c24919e40eaa098fb6d75923324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008146610ce442fa16997c70196f130" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:25.836642 26902 mvcc.cc:205] Tried to move back new op lower bound from 7333913992503668736 to 7333913992436355072. Current Snapshot: MvccSnapshot[applied={T|T < 7333913992503668736}]
I20260927 10:52:25.837517 26901 mvcc.cc:205] Tried to move back new op lower bound from 7333913992503668736 to 7333913992436355072. Current Snapshot: MvccSnapshot[applied={T|T < 7333913992503668736}]
I20260927 10:52:25.845111 26411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:25.845252 26411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:25.847674 26411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83edd443af7a4ca2871f9cbb386fb903: Sending DeleteTablet for 3 replicas of tablet dcbcb4a91ce44549aa03b2dcaa704bce
I20260927 10:52:25.848402 26670 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbcb4a91ce44549aa03b2dcaa704bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:25 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:46198
I20260927 10:52:25.848493 26804 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbcb4a91ce44549aa03b2dcaa704bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:25 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:36342
I20260927 10:52:25.848491 26536 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbcb4a91ce44549aa03b2dcaa704bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:25 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:45576
I20260927 10:52:25.849154 26910 tablet_replica.cc:330] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: stopping tablet replica
I20260927 10:52:25.849431 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26473
I20260927 10:52:25.849468 26910 raft_consensus.cc:2229] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:25.851176 26910 raft_consensus.cc:2258] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:25.851557 26911 tablet_replica.cc:330] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: stopping tablet replica
I20260927 10:52:25.851872 26911 raft_consensus.cc:2229] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:25.852171 26911 raft_consensus.cc:2258] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:25.852628 26910 ts_tablet_manager.cc:1905] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:25.853277 26911 ts_tablet_manager.cc:1905] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:25.855683 26911 ts_tablet_manager.cc:1918] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:25.855792 26911 log.cc:1196] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/dcbcb4a91ce44549aa03b2dcaa704bce
I20260927 10:52:25.856078 26911 ts_tablet_manager.cc:1939] T dcbcb4a91ce44549aa03b2dcaa704bce P e008146610ce442fa16997c70196f130: Deleting consensus metadata
I20260927 10:52:25.856161 26910 ts_tablet_manager.cc:1918] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:25.856266 26910 log.cc:1196] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/dcbcb4a91ce44549aa03b2dcaa704bce
I20260927 10:52:25.856597 26910 ts_tablet_manager.cc:1939] T dcbcb4a91ce44549aa03b2dcaa704bce P a1c5527276094ac3a10d111fc1d8b1e3: Deleting consensus metadata
I20260927 10:52:25.857291 26398 catalog_manager.cc:4841] TS e008146610ce442fa16997c70196f130 (127.9.39.131:42807): tablet dcbcb4a91ce44549aa03b2dcaa704bce (table test-table [id=bab2d9bb5f214b0fa3199dd993a65f6e]) successfully deleted
I20260927 10:52:25.857736 26397 catalog_manager.cc:4841] TS a1c5527276094ac3a10d111fc1d8b1e3 (127.9.39.130:34997): tablet dcbcb4a91ce44549aa03b2dcaa704bce (table test-table [id=bab2d9bb5f214b0fa3199dd993a65f6e]) successfully deleted
I20260927 10:52:25.858482 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26607
W20260927 10:52:25.858459 26395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:42185 (error 108)
I20260927 10:52:25.860069 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:25.859746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:42185 (local address 127.0.0.1:45602)
0927 10:52:25.859893 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:25.859970 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 10:52:25.860234 26395 catalog_manager.cc:4584] TS d3429c24919e40eaa098fb6d75923324 (127.9.39.129:42185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dcbcb4a91ce44549aa03b2dcaa704bce: Network error: Client connection negotiation failed: client connection to 127.9.39.129:42185: connect: Connection refused (error 111)
I20260927 10:52:25.864424 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26741
I20260927 10:52:25.871457 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26379
2026-09-27T10:52:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2199 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T10:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:25Z Disabled control of system clock
I20260927 10:52:25.901217 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39329
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 10:52:26.010303 26917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:26.010591 26917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:26.010627 26917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:26.014727 26917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:26.014796 26917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:26.014811 26917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:26.014830 26917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:26.014846 26917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:26.019832 26917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:26.020999 26917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:26.022409 26917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:26.030915 26922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.030910 26925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.030910 26923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:26.031118 26917 server_base.cc:1034] running on GCE node
I20260927 10:52:26.031847 26917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:26.032910 26917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:26.034071 26917 hybrid_clock.cc:648] HybridClock initialized: now 1790506346034051 us; error 32 us; skew 500 ppm
I20260927 10:52:26.035995 26917 webserver.cc:460] Webserver started at http://127.9.39.190:33161/ using document root <none> and password file <none>
I20260927 10:52:26.036566 26917 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:26.036628 26917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:26.036850 26917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:26.039157 26917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "a2e733ccb5e74ac1989b4ce32082f0b0"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "78d2556f88ea6e3c68cc1a9f623eadf2"
server_key_iv: "e0437ea4c8d4a7253b5104e77360da8b"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.039727 26917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "a2e733ccb5e74ac1989b4ce32082f0b0"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "78d2556f88ea6e3c68cc1a9f623eadf2"
server_key_iv: "e0437ea4c8d4a7253b5104e77360da8b"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.043572 26917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:52:26.046113 26931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.047435 26917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:26.047559 26917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "a2e733ccb5e74ac1989b4ce32082f0b0"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "78d2556f88ea6e3c68cc1a9f623eadf2"
server_key_iv: "e0437ea4c8d4a7253b5104e77360da8b"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.047701 26917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:26.062148 26917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:26.062956 26917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:26.063160 26917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:26.070533 26917 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39329
I20260927 10:52:26.070536 26983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39329 every 8 connection(s)
I20260927 10:52:26.071569 26917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:26.074760 26984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:26.076893 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 26917
I20260927 10:52:26.077009 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:26.077313 9374 external_mini_cluster.cc:1428] Setting key 52f87f45a2c0441642e630b5481487d8
I20260927 10:52:26.081053 26984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0: Bootstrap starting.
I20260927 10:52:26.083336 26984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:26.084072 26984 log.cc:824] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:26.086143 26984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0: No bootstrap required, opened a new log
I20260927 10:52:26.086797 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.078769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.9.39.190:39329)
0927 10:52:26.079414 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.079430 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.079464 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:52:26.080244 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.080263 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.080677 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.081018 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:26.081464 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.082426 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.083380 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.083911 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.084074 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.084969 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:26.084974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:26.084994 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:26.085381 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:26.085436 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:26.085455 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:26.085561 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.085709 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.085970 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":117,"threads_started":1}
I20260927 10:52:26.090072 26984 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39329 } }
I20260927 10:52:26.090334 26984 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:26.090408 26984 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e733ccb5e74ac1989b4ce32082f0b0, State: Initialized, Role: FOLLOWER
I20260927 10:52:26.090876 26984 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39329 } }
I20260927 10:52:26.091023 26984 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:26.091110 26984 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:26.091217 26984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:26.092164 26984 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39329 } }
I20260927 10:52:26.092538 26984 leader_election.cc:302] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2e733ccb5e74ac1989b4ce32082f0b0; no voters:
I20260927 10:52:26.092900 26984 leader_election.cc:288] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:26.092998 26989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:26.093210 26989 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [term 1 LEADER]: Becoming Leader. State: Replica: a2e733ccb5e74ac1989b4ce32082f0b0, State: Running, Role: LEADER
I20260927 10:52:26.093504 26989 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39329 } }
I20260927 10:52:26.094108 26984 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:26.095139 26990 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39329 } } }
I20260927 10:52:26.095273 26990 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:26.095353 26991 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2e733ccb5e74ac1989b4ce32082f0b0. Latest consensus state: current_term: 1 leader_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e733ccb5e74ac1989b4ce32082f0b0" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39329 } } }
I20260927 10:52:26.095456 26991 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:26.095687 26998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:26.098836 26998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:26.104454 26998 catalog_manager.cc:1347] Generated new cluster ID: fd21e725011b489d9ee850fa1d9a2afa
I20260927 10:52:26.104523 26998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:26.118479 26998 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:26.119446 26998 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:26.134295 26998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0: Generated new TSK 0
I20260927 10:52:26.135179 26998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:26.137938 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39329
--builtin_ntp_servers=127.9.39.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 10:52:26.247330 27008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:26.247573 27008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:26.247642 27008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:26.251677 27008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:26.251730 27008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:26.251808 27008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:26.256668 27008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:26.257714 27008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:26.259104 27008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:26.268301 27016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:26.268417 27008 server_base.cc:1034] running on GCE node
W20260927 10:52:26.268277 27013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.268610 27014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:26.268983 27008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:26.269618 27008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:26.270761 27008 hybrid_clock.cc:648] HybridClock initialized: now 1790506346270745 us; error 37 us; skew 500 ppm
I20260927 10:52:26.272533 27008 webserver.cc:460] Webserver started at http://127.9.39.129:37259/ using document root <none> and password file <none>
I20260927 10:52:26.273111 27008 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:26.273201 27008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:26.273412 27008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:26.275408 27008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "232224c95c2a468881171da62445f4b9"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "086ed8156a637ed0640e34a4200ec1ce"
server_key_iv: "9acc32d40aeeff78574b0f4ed37ebd1d"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.275941 27008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "232224c95c2a468881171da62445f4b9"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "086ed8156a637ed0640e34a4200ec1ce"
server_key_iv: "9acc32d40aeeff78574b0f4ed37ebd1d"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.279451 27008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:52:26.281702 27022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.282910 27008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260927 10:52:26.283023 27008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "232224c95c2a468881171da62445f4b9"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "086ed8156a637ed0640e34a4200ec1ce"
server_key_iv: "9acc32d40aeeff78574b0f4ed37ebd1d"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.283133 27008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:26.297128 27008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:26.297830 27008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:26.298025 27008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:26.298605 27008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:26.299553 27008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:26.299621 27008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.299686 27008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:26.299733 27008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.308574 27008 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:40401
I20260927 10:52:26.308605 27135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:40401 every 8 connection(s)
I20260927 10:52:26.309453 27008 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:26.314548 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27008
I20260927 10:52:26.314705 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:52:26.315008 9374 external_mini_cluster.cc:1428] Setting key 2244f23f404954fa4e241e8e0a24ebe4
I20260927 10:52:26.316989 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39329
--builtin_ntp_servers=127.9.39.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 10:52:26.319504 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.311318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:44105 (local address 127.9.39.190:39329)
0927 10:52:26.311465 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.311470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.312589 (+ 1119us) server_negotiation.cc:407] Connection header received
0927 10:52:26.313929 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.313932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.313979 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.314055 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:26.315058 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.315718 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.317296 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.317506 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.317597 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.318970 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:26.318977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:26.318981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:26.318997 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:26.319024 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:26.319035 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:26.319124 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.319259 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.319331 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:52:26.320277 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.311767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39329 (local address 127.9.39.129:44105)
0927 10:52:26.312441 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.312483 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.313730 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.314224 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.314231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.314317 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:26.314915 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.314933 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.315855 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.315857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.317177 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.317190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.317689 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.317696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.317930 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.318664 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:26.318666 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:26.318744 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:26.318815 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:26.319122 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:26.319129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:26.319133 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.319229 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.319445 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":116,"threads_started":1}
I20260927 10:52:26.321419 27136 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39329
I20260927 10:52:26.321640 27136 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:26.322147 27136 heartbeater.cc:511] Master 127.9.39.190:39329 requested a full tablet report, sending...
I20260927 10:52:26.323556 26948 ts_manager.cc:194] Registered new tserver with Master: 232224c95c2a468881171da62445f4b9 (127.9.39.129:40401)
I20260927 10:52:26.324681 26948 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.129:44105
W20260927 10:52:26.430733 27139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:26.431021 27139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:26.431087 27139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:26.435037 27139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:26.435113 27139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:26.435201 27139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:26.439797 27139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:26.440796 27139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:26.442155 27139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:26.452350 27147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.452315 27145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.452327 27144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:26.452621 27139 server_base.cc:1034] running on GCE node
I20260927 10:52:26.453065 27139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:26.453655 27139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:26.454815 27139 hybrid_clock.cc:648] HybridClock initialized: now 1790506346454781 us; error 43 us; skew 500 ppm
I20260927 10:52:26.456789 27139 webserver.cc:460] Webserver started at http://127.9.39.130:40663/ using document root <none> and password file <none>
I20260927 10:52:26.457338 27139 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:26.457425 27139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:26.457648 27139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:26.459650 27139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "3a8cd6206b2b4a3590e9bd56e8a36955"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "a8445fc08bbde9aee81e89ae35e5959d"
server_key_iv: "843a3672ebf2bf72ce9ae750bfab536d"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.460192 27139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "3a8cd6206b2b4a3590e9bd56e8a36955"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "a8445fc08bbde9aee81e89ae35e5959d"
server_key_iv: "843a3672ebf2bf72ce9ae750bfab536d"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.463832 27139 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:52:26.466259 27153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.467635 27139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260927 10:52:26.467767 27139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "3a8cd6206b2b4a3590e9bd56e8a36955"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "a8445fc08bbde9aee81e89ae35e5959d"
server_key_iv: "843a3672ebf2bf72ce9ae750bfab536d"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.467876 27139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:26.490221 27139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:26.490965 27139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:26.491163 27139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:26.491751 27139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:26.492681 27139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:26.492748 27139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.492830 27139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:26.492872 27139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.501940 27139 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:44765
I20260927 10:52:26.501969 27266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:44765 every 8 connection(s)
I20260927 10:52:26.502888 27139 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:52:26.503615 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27139
I20260927 10:52:26.503752 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:52:26.504042 9374 external_mini_cluster.cc:1428] Setting key 826e75eaa197c384c234a3841fcfbfb7
I20260927 10:52:26.506110 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39329
--builtin_ntp_servers=127.9.39.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 10:52:26.512275 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.504792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:45893 (local address 127.9.39.190:39329)
0927 10:52:26.504917 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.504922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.506072 (+ 1150us) server_negotiation.cc:407] Connection header received
0927 10:52:26.507574 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.507577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.507628 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.507707 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:26.508608 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.509143 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.510287 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.510535 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.510643 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.511784 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:26.511788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:26.511791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:26.511804 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:26.511828 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:26.511837 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:26.511907 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.512068 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.512131 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:52:26.513100 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.505270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39329 (local address 127.9.39.130:45893)
0927 10:52:26.505939 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.505978 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.507376 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.507872 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.507880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.507965 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:26.508439 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.508451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.509267 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.509272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.510168 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.510177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.510684 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.510714 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.510868 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.511504 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:26.511506 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:26.511576 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:26.511640 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:26.511920 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:26.511930 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:26.511941 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.512061 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.512341 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":137,"threads_started":1}
I20260927 10:52:26.514433 27267 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39329
I20260927 10:52:26.514672 27267 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:26.515237 27267 heartbeater.cc:511] Master 127.9.39.190:39329 requested a full tablet report, sending...
I20260927 10:52:26.516350 26948 ts_manager.cc:194] Registered new tserver with Master: 3a8cd6206b2b4a3590e9bd56e8a36955 (127.9.39.130:44765)
I20260927 10:52:26.516937 26948 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.130:45893
W20260927 10:52:26.614856 27270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:26.615114 27270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:26.615175 27270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:26.619148 27270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:26.619217 27270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:26.619300 27270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:26.624029 27270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:26.625042 27270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:26.626370 27270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:52:26.635627 27270 server_base.cc:1034] running on GCE node
W20260927 10:52:26.635519 27276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.635509 27278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:26.635805 27275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:26.636245 27270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:26.637091 27270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:26.638269 27270 hybrid_clock.cc:648] HybridClock initialized: now 1790506346638258 us; error 30 us; skew 500 ppm
I20260927 10:52:26.640033 27270 webserver.cc:460] Webserver started at http://127.9.39.131:42019/ using document root <none> and password file <none>
I20260927 10:52:26.640585 27270 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:26.640641 27270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:26.640865 27270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:26.642817 27270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "64dd14f73d844539a4cc9b39748b64a6"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "30b9b31f92fedf1b689f50a47ce4e337"
server_key_iv: "fc9d39f443fdab73dba9837105cba56a"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.643339 27270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "64dd14f73d844539a4cc9b39748b64a6"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "30b9b31f92fedf1b689f50a47ce4e337"
server_key_iv: "fc9d39f443fdab73dba9837105cba56a"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.646893 27270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:52:26.649314 27284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.650509 27270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:26.650663 27270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "64dd14f73d844539a4cc9b39748b64a6"
format_stamp: "Formatted at 2026-09-27 10:52:26 on dist-test-slave-p9q6"
server_key: "30b9b31f92fedf1b689f50a47ce4e337"
server_key_iv: "fc9d39f443fdab73dba9837105cba56a"
server_key_version: "encryptionkey@0"
I20260927 10:52:26.650806 27270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:26.679169 27270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:26.679868 27270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:26.680037 27270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:26.680641 27270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:26.681582 27270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:26.681627 27270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.681667 27270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:26.681716 27270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:26.690519 27270 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:42325
I20260927 10:52:26.690588 27397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:42325 every 8 connection(s)
I20260927 10:52:26.691474 27270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:26.692085 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27270
I20260927 10:52:26.692195 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:52:26.692479 9374 external_mini_cluster.cc:1428] Setting key 1a939935b8d4f53142b57a8e56cec91d
I20260927 10:52:26.700980 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.693419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:39931 (local address 127.9.39.190:39329)
0927 10:52:26.693543 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.693548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.694951 (+ 1403us) server_negotiation.cc:407] Connection header received
0927 10:52:26.696350 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.696352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.696395 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.696509 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:26.697465 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.697976 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.699222 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.699426 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.699517 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.700552 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:26.700555 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:26.700557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:26.700568 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:26.700586 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:26.700594 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:26.700646 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.700824 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.700872 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:52:26.701853 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.693964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:39329 (local address 127.9.39.131:39931)
0927 10:52:26.694827 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.694868 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.696180 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.696721 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.696728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.696810 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:26.697326 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.697341 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.698112 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.698115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.699110 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.699120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.699594 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.699597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.699744 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.700308 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:26.700310 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:26.700349 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:26.700411 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:26.700695 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:26.700702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:26.700707 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.700816 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.701084 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":135,"threads_started":1}
I20260927 10:52:26.702939 27398 heartbeater.cc:347] Connected to a master server at 127.9.39.190:39329
I20260927 10:52:26.703163 27398 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:26.703650 27398 heartbeater.cc:511] Master 127.9.39.190:39329 requested a full tablet report, sending...
I20260927 10:52:26.704573 26948 ts_manager.cc:194] Registered new tserver with Master: 64dd14f73d844539a4cc9b39748b64a6 (127.9.39.131:42325)
I20260927 10:52:26.705111 26948 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.131:39931
I20260927 10:52:26.706313 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:52:26.711663 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.708339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.9.39.190:39329)
0927 10:52:26.708450 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.708454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.708623 (+ 169us) server_negotiation.cc:407] Connection header received
0927 10:52:26.708704 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.708706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.708739 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.708801 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:26.709031 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.709505 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.710282 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.710476 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.710576 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.711339 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:26.711341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:26.711343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:26.711353 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:26.711370 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:26.711377 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:26.711432 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.711500 (+ 68us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.711566 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:52:26.714244 26948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:52:26.717108 26948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:52:26.730515 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.725660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40401 (local address 127.0.0.1:53190)
0927 10:52:26.725994 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.726006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.726100 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.726733 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.726737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.726754 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:26.726841 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.726849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.727796 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.727802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.729121 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.729131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.730009 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.730013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.730112 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.730147 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.730246 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.730359 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":39,"spinlock_wait_cycles":8832,"thread_start_us":80,"threads_started":1}
I20260927 10:52:26.730515 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.725834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.9.39.129:40401)
0927 10:52:26.726320 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.726331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.726351 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:52:26.726418 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.726422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.726584 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.726753 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:26.726966 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.727691 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.729264 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.729889 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.730008 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.730214 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.730270 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.730360 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":93,"threads_started":1}
I20260927 10:52:26.730927 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.725134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:44765 (local address 127.0.0.1:59896)
0927 10:52:26.725973 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.725986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.726096 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.726851 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.726854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.726871 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:26.726946 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.726952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.728377 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.728381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.729393 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.729402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.730338 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.730341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.730433 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.730457 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.730507 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.730568 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"spinlock_wait_cycles":8320,"thread_start_us":116,"threads_started":1}
I20260927 10:52:26.731557 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.725283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.9.39.130:44765)
0927 10:52:26.726375 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.726383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.726403 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:52:26.726460 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.726464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.726664 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.727086 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:26.727209 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.728239 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.729512 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.730215 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.731108 (+ 893us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.731265 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.731310 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.731391 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1011,"thread_start_us":121,"threads_started":1}
I20260927 10:52:26.731792 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.724870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:42325 (local address 127.0.0.1:40786)
0927 10:52:26.725607 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.725654 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.725786 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.726548 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.726553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.726576 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:26.726732 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.726741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.728162 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.728166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.729275 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.729285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.730504 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.730507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.731485 (+ 978us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.731508 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.731596 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.731672 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260927 10:52:26.732275 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.725112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.9.39.131:42325)
0927 10:52:26.725579 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.725593 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.726098 (+ 505us) server_negotiation.cc:407] Connection header received
0927 10:52:26.726180 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.726187 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.726395 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.726551 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:26.726865 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.728043 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.729386 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.730157 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.730272 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.730446 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.732020 (+ 1574us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.732111 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":141,"threads_started":1}
I20260927 10:52:26.733675 27070 tablet_service.cc:1467] Processing CreateTablet for tablet b8cc107a23e04dc087b6813bf23245f6 (DEFAULT_TABLE table=test-table [id=81df68c743284c4d8c50f0b9d9fafb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:26.734164 27332 tablet_service.cc:1467] Processing CreateTablet for tablet b8cc107a23e04dc087b6813bf23245f6 (DEFAULT_TABLE table=test-table [id=81df68c743284c4d8c50f0b9d9fafb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:26.734095 27201 tablet_service.cc:1467] Processing CreateTablet for tablet b8cc107a23e04dc087b6813bf23245f6 (DEFAULT_TABLE table=test-table [id=81df68c743284c4d8c50f0b9d9fafb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:26.734791 27070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8cc107a23e04dc087b6813bf23245f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:26.735054 27332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8cc107a23e04dc087b6813bf23245f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:26.735060 27201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8cc107a23e04dc087b6813bf23245f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:26.740800 27415 tablet_bootstrap.cc:492] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Bootstrap starting.
I20260927 10:52:26.742240 27417 tablet_bootstrap.cc:492] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: Bootstrap starting.
I20260927 10:52:26.742235 27416 tablet_bootstrap.cc:492] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Bootstrap starting.
I20260927 10:52:26.743093 27415 tablet_bootstrap.cc:654] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:26.743785 27415 log.cc:824] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:26.744587 27416 tablet_bootstrap.cc:654] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:26.744587 27417 tablet_bootstrap.cc:654] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:26.745460 27416 log.cc:824] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:26.745460 27417 log.cc:824] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:26.745864 27415 tablet_bootstrap.cc:492] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: No bootstrap required, opened a new log
I20260927 10:52:26.746279 27415 ts_tablet_manager.cc:1397] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:52:26.747488 27417 tablet_bootstrap.cc:492] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: No bootstrap required, opened a new log
I20260927 10:52:26.747697 27417 ts_tablet_manager.cc:1397] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:52:26.747737 27416 tablet_bootstrap.cc:492] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: No bootstrap required, opened a new log
I20260927 10:52:26.747931 27416 ts_tablet_manager.cc:1397] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 10:52:26.750595 27415 raft_consensus.cc:348] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.750880 27415 raft_consensus.cc:374] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:26.750937 27415 raft_consensus.cc:729] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64dd14f73d844539a4cc9b39748b64a6, State: Initialized, Role: FOLLOWER
I20260927 10:52:26.751283 27416 raft_consensus.cc:348] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.751412 27415 consensus_queue.cc:260] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.751505 27416 raft_consensus.cc:374] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:26.751554 27416 raft_consensus.cc:729] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a8cd6206b2b4a3590e9bd56e8a36955, State: Initialized, Role: FOLLOWER
I20260927 10:52:26.751844 27417 raft_consensus.cc:348] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.752075 27417 raft_consensus.cc:374] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:26.751991 27416 consensus_queue.cc:260] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.752120 27417 raft_consensus.cc:729] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232224c95c2a468881171da62445f4b9, State: Initialized, Role: FOLLOWER
I20260927 10:52:26.752174 27398 heartbeater.cc:503] Master 127.9.39.190:39329 was elected leader, sending a full tablet report...
I20260927 10:52:26.752589 27415 ts_tablet_manager.cc:1428] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260927 10:52:26.752660 27417 consensus_queue.cc:260] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.752866 27416 ts_tablet_manager.cc:1428] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:52:26.753171 27267 heartbeater.cc:503] Master 127.9.39.190:39329 was elected leader, sending a full tablet report...
I20260927 10:52:26.753453 27417 ts_tablet_manager.cc:1428] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260927 10:52:26.753763 27268 tablet.cc:2369] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:52:26.753993 27136 heartbeater.cc:503] Master 127.9.39.190:39329 was elected leader, sending a full tablet report...
W20260927 10:52:26.810679 27137 tablet.cc:2369] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:52:26.890479 27421 raft_consensus.cc:482] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:26.890801 27421 raft_consensus.cc:504] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.892028 27421 leader_election.cc:288] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a8cd6206b2b4a3590e9bd56e8a36955 (127.9.39.130:44765), 232224c95c2a468881171da62445f4b9 (127.9.39.129:40401)
I20260927 10:52:26.896095 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.892383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:59261 (local address 127.9.39.129:40401)
0927 10:52:26.892511 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.892516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.892538 (+ 22us) server_negotiation.cc:407] Connection header received
0927 10:52:26.892631 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.892635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.892690 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.892803 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:26.893102 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.893821 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.895039 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.895497 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.895631 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.895687 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.895883 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.895949 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:52:26.896095 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.892284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40401 (local address 127.9.39.131:59261)
0927 10:52:26.892383 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.892404 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.892506 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.892810 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.892812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.892828 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:26.892979 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.892986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.893930 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.893935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.894897 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.894907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.895623 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.895627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.895731 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.895760 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.895855 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.895934 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 10:52:26.896135 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.892382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:50167 (local address 127.9.39.130:44765)
0927 10:52:26.892521 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.892528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.892893 (+ 365us) server_negotiation.cc:407] Connection header received
0927 10:52:26.893026 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.893031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.893106 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.893163 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:26.893463 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.894038 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.895169 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.895543 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.895672 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.895719 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.895950 (+ 231us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.896020 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:52:26.896117 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.892284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:44765 (local address 127.9.39.131:50167)
0927 10:52:26.892790 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.892806 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:26.892912 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:26.893200 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:26.893203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:26.893222 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:26.893351 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.893358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.894154 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.894159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.895058 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:26.895067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.895672 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.895676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:26.895767 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.895787 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 10:52:26.895874 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 10:52:26.895951 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":111,"thread_start_us":125,"threads_started":1}
I20260927 10:52:26.896899 27090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cc107a23e04dc087b6813bf23245f6" candidate_uuid: "64dd14f73d844539a4cc9b39748b64a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232224c95c2a468881171da62445f4b9" is_pre_election: true
I20260927 10:52:26.896908 27221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cc107a23e04dc087b6813bf23245f6" candidate_uuid: "64dd14f73d844539a4cc9b39748b64a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" is_pre_election: true
I20260927 10:52:26.897231 27090 raft_consensus.cc:2454] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64dd14f73d844539a4cc9b39748b64a6 in term 0.
I20260927 10:52:26.897231 27221 raft_consensus.cc:2454] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64dd14f73d844539a4cc9b39748b64a6 in term 0.
I20260927 10:52:26.897797 27287 leader_election.cc:302] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a8cd6206b2b4a3590e9bd56e8a36955, 64dd14f73d844539a4cc9b39748b64a6; no voters:
I20260927 10:52:26.898115 27421 raft_consensus.cc:2781] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:52:26.898206 27421 raft_consensus.cc:482] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:52:26.898269 27421 raft_consensus.cc:3037] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:26.899384 27421 raft_consensus.cc:504] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.899778 27421 leader_election.cc:288] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [CANDIDATE]: Term 1 election: Requested vote from peers 3a8cd6206b2b4a3590e9bd56e8a36955 (127.9.39.130:44765), 232224c95c2a468881171da62445f4b9 (127.9.39.129:40401)
I20260927 10:52:26.900111 27221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cc107a23e04dc087b6813bf23245f6" candidate_uuid: "64dd14f73d844539a4cc9b39748b64a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955"
I20260927 10:52:26.900184 27090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cc107a23e04dc087b6813bf23245f6" candidate_uuid: "64dd14f73d844539a4cc9b39748b64a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232224c95c2a468881171da62445f4b9"
I20260927 10:52:26.900230 27221 raft_consensus.cc:3037] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:26.900355 27090 raft_consensus.cc:3037] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:26.901329 27221 raft_consensus.cc:2454] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64dd14f73d844539a4cc9b39748b64a6 in term 1.
I20260927 10:52:26.901369 27090 raft_consensus.cc:2454] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64dd14f73d844539a4cc9b39748b64a6 in term 1.
I20260927 10:52:26.901640 27287 leader_election.cc:302] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a8cd6206b2b4a3590e9bd56e8a36955, 64dd14f73d844539a4cc9b39748b64a6; no voters:
I20260927 10:52:26.901823 27421 raft_consensus.cc:2781] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:26.902063 27421 raft_consensus.cc:686] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 1 LEADER]: Becoming Leader. State: Replica: 64dd14f73d844539a4cc9b39748b64a6, State: Running, Role: LEADER
I20260927 10:52:26.902403 27421 consensus_queue.cc:237] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } }
I20260927 10:52:26.905450 26946 catalog_manager.cc:5495] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64dd14f73d844539a4cc9b39748b64a6 (127.9.39.131). New cstate: current_term: 1 leader_uuid: "64dd14f73d844539a4cc9b39748b64a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dd14f73d844539a4cc9b39748b64a6" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 42325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 } health_report { overall_health: UNKNOWN } } }
I20260927 10:52:26.917871 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.914328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.9.39.131:42325)
0927 10:52:26.914445 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:26.914452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:26.914497 (+ 45us) server_negotiation.cc:407] Connection header received
0927 10:52:26.914738 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:26.914741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:26.914799 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:26.914900 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:52:26.915129 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.915623 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.916478 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:26.916697 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:26.916802 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:26.917092 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:52:26.917476 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:52:26.917602 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:26.917668 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 10:52:26.917715 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:52:26.923431 27090 raft_consensus.cc:1261] T b8cc107a23e04dc087b6813bf23245f6 P 232224c95c2a468881171da62445f4b9 [term 1 FOLLOWER]: Refusing update from remote peer 64dd14f73d844539a4cc9b39748b64a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:26.923432 27221 raft_consensus.cc:1261] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 1 FOLLOWER]: Refusing update from remote peer 64dd14f73d844539a4cc9b39748b64a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:26.924084 27421 consensus_queue.cc:1035] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "232224c95c2a468881171da62445f4b9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 40401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:26.924270 27425 consensus_queue.cc:1035] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a8cd6206b2b4a3590e9bd56e8a36955" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 44765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:26.939834 26948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:26.939970 26948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:26.942082 26948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2e733ccb5e74ac1989b4ce32082f0b0: Sending DeleteTablet for 3 replicas of tablet b8cc107a23e04dc087b6813bf23245f6
I20260927 10:52:26.942785 27332 tablet_service.cc:1514] Processing DeleteTablet for tablet b8cc107a23e04dc087b6813bf23245f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:26 UTC) from {username='slave'} at 127.0.0.1:40786
I20260927 10:52:26.942804 27070 tablet_service.cc:1514] Processing DeleteTablet for tablet b8cc107a23e04dc087b6813bf23245f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:26 UTC) from {username='slave'} at 127.0.0.1:53190
I20260927 10:52:26.942852 27201 tablet_service.cc:1514] Processing DeleteTablet for tablet b8cc107a23e04dc087b6813bf23245f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:26 UTC) from {username='slave'} at 127.0.0.1:59896
I20260927 10:52:26.943444 27440 tablet_replica.cc:330] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: stopping tablet replica
I20260927 10:52:26.943735 27440 raft_consensus.cc:2229] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:26.944020 27440 raft_consensus.cc:2258] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:26.944116 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27008
I20260927 10:52:26.944736 27438 tablet_replica.cc:330] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: stopping tablet replica
I20260927 10:52:26.945009 27438 raft_consensus.cc:2229] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:52:26.945361 27438 raft_consensus.cc:2258] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:26.947909 27440 ts_tablet_manager.cc:1905] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:26.947939 27438 ts_tablet_manager.cc:1905] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:26.950721 27440 ts_tablet_manager.cc:1918] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:26.950878 27440 log.cc:1196] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/b8cc107a23e04dc087b6813bf23245f6
I20260927 10:52:26.951242 27440 ts_tablet_manager.cc:1939] T b8cc107a23e04dc087b6813bf23245f6 P 3a8cd6206b2b4a3590e9bd56e8a36955: Deleting consensus metadata
W20260927 10:52:26.951208 26935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:40401 (error 108)
I20260927 10:52:26.951325 27438 ts_tablet_manager.cc:1918] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:26.951416 27438 log.cc:1196] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/b8cc107a23e04dc087b6813bf23245f6
I20260927 10:52:26.951766 27438 ts_tablet_manager.cc:1939] T b8cc107a23e04dc087b6813bf23245f6 P 64dd14f73d844539a4cc9b39748b64a6: Deleting consensus metadata
I20260927 10:52:26.952028 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27139
I20260927 10:52:26.953409 26933 catalog_manager.cc:4841] TS 64dd14f73d844539a4cc9b39748b64a6 (127.9.39.131:42325): tablet b8cc107a23e04dc087b6813bf23245f6 (table test-table [id=81df68c743284c4d8c50f0b9d9fafb2b]) successfully deleted
I20260927 10:52:26.959014 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27270
I20260927 10:52:26.960307 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.959951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:40401 (local address 127.0.0.1:53192)
0927 10:52:26.960127 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.960204 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260927 10:52:26.960417 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:26.960097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:44765 (local address 127.0.0.1:59900)
0927 10:52:26.960228 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:26.960293 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.130:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 10:52:26.960600 26934 catalog_manager.cc:4584] TS 3a8cd6206b2b4a3590e9bd56e8a36955 (127.9.39.130:44765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8cc107a23e04dc087b6813bf23245f6: Network error: Client connection negotiation failed: client connection to 127.9.39.130:44765: connect: Connection refused (error 111)
I20260927 10:52:26.965140 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 26917
2026-09-27T10:52:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1092 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:27 dist-test-slave-p9q6 krb5kdc[27444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:27 dist-test-slave-p9q6 krb5kdc[27444](info): set up 2 sockets
Sep 27 10:52:27 dist-test-slave-p9q6 krb5kdc[27444](info): commencing operation
krb5kdc: starting...
W20260927 10:52:27.998446 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:28.121631 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36751
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:36751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:28.228415 27457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:28.228718 27457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:28.228788 27457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:28.232755 27457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:28.232851 27457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:28.232890 27457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:28.232941 27457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:28.232977 27457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:28.238230 27457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:36751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:36751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:28.239315 27457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:28.240374 27457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:28.248772 27463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:28.248725 27465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:28.248803 27462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:28.248854 27457 server_base.cc:1034] running on GCE node
I20260927 10:52:28.249547 27457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:28.250650 27457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:28.251852 27457 hybrid_clock.cc:648] HybridClock initialized: now 1790506348251833 us; error 28 us; skew 500 ppm
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:28.254578 27457 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:28.255479 27457 webserver.cc:460] Webserver started at http://127.9.39.190:41915/ using document root <none> and password file <none>
I20260927 10:52:28.256026 27457 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:28.256085 27457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:28.256311 27457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:28.258302 27457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "4e3826829e264e96bd59ac526c7dee84"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "94c03c73d5dc4e9af5ba1d9a6edf5383"
server_key_iv: "d0504176de30b87996dddf3f41fd76f9"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.258886 27457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "4e3826829e264e96bd59ac526c7dee84"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "94c03c73d5dc4e9af5ba1d9a6edf5383"
server_key_iv: "d0504176de30b87996dddf3f41fd76f9"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.262421 27457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 10:52:28.264972 27472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.266348 27457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260927 10:52:28.266475 27457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "4e3826829e264e96bd59ac526c7dee84"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "94c03c73d5dc4e9af5ba1d9a6edf5383"
server_key_iv: "d0504176de30b87996dddf3f41fd76f9"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.266580 27457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:28.278978 27457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:28.282076 27457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:28.282271 27457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:28.289098 27457 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:36751
I20260927 10:52:28.289124 27524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:36751 every 8 connection(s)
I20260927 10:52:28.290017 27457 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:28.292654 27525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:28.297667 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27457
I20260927 10:52:28.297834 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:28.298120 9374 external_mini_cluster.cc:1428] Setting key beea1659fff664b0df9037b044f579a9
I20260927 10:52:28.298848 27525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84: Bootstrap starting.
I20260927 10:52:28.301371 27525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:28.302135 27525 log.cc:824] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:28.304100 27525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84: No bootstrap required, opened a new log
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:28.308291 27525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3826829e264e96bd59ac526c7dee84" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36751 } }
I20260927 10:52:28.308562 27525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:28.308640 27525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3826829e264e96bd59ac526c7dee84, State: Initialized, Role: FOLLOWER
I20260927 10:52:28.309131 27525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3826829e264e96bd59ac526c7dee84" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36751 } }
I20260927 10:52:28.309541 27525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:28.309630 27525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:28.309746 27525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:28.311005 27525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3826829e264e96bd59ac526c7dee84" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36751 } }
I20260927 10:52:28.311391 27525 leader_election.cc:302] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e3826829e264e96bd59ac526c7dee84; no voters:
I20260927 10:52:28.311800 27525 leader_election.cc:288] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:28.311933 27530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:28.312181 27530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [term 1 LEADER]: Becoming Leader. State: Replica: 4e3826829e264e96bd59ac526c7dee84, State: Running, Role: LEADER
I20260927 10:52:28.312494 27530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3826829e264e96bd59ac526c7dee84" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36751 } }
I20260927 10:52:28.313053 27525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:28.313196 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:28.299655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.9.39.190:36751)
0927 10:52:28.300092 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:28.300107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:28.300142 (+ 35us) server_negotiation.cc:407] Connection header received
0927 10:52:28.300778 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:28.300795 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:28.301146 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:28.301484 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:28.302303 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.303343 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.304184 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.304655 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.304816 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:28.307666 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:28.307688 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:28.307709 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:28.307749 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:28.309814 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:28.310317 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:28.310322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:28.310326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:28.310435 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:28.310749 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:28.310752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:28.310754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:28.311211 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:28.311361 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:28.311714 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 10:52:28.312226 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":130,"threads_started":1}
I20260927 10:52:28.313683 27530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e3826829e264e96bd59ac526c7dee84. Latest consensus state: current_term: 1 leader_uuid: "4e3826829e264e96bd59ac526c7dee84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3826829e264e96bd59ac526c7dee84" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36751 } } }
I20260927 10:52:28.313791 27530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:28.314390 27531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e3826829e264e96bd59ac526c7dee84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3826829e264e96bd59ac526c7dee84" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 36751 } } }
I20260927 10:52:28.314546 27531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:28.315853 27536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:28.318854 27536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:28.324465 27536 catalog_manager.cc:1347] Generated new cluster ID: 03ce033be45443f291e1fe616983ce18
I20260927 10:52:28.324540 27536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:28.337116 27536 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:28.338111 27536 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:28.345975 27536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84: Generated new TSK 0
I20260927 10:52:28.347056 27536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:52:28.382001 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36751
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:28.548525 27551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:28.548811 27551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:28.548859 27551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:28.552860 27551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:28.552924 27551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:28.552985 27551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:28.558156 27551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:28.559240 27551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:28.560287 27551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:28.569885 27557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:28.569856 27556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:28.569924 27551 server_base.cc:1034] running on GCE node
W20260927 10:52:28.569872 27559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:28.570624 27551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:28.571290 27551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:28.572506 27551 hybrid_clock.cc:648] HybridClock initialized: now 1790506348572464 us; error 30 us; skew 500 ppm
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:28.575273 27551 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:52:28.576237 27551 webserver.cc:460] Webserver started at http://127.9.39.129:37567/ using document root <none> and password file <none>
I20260927 10:52:28.576802 27551 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:28.576905 27551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:28.577152 27551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:28.579304 27551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "68811b9ab94f4d10ad1a723f1f425f50"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "c48be6b4af034cd336951cd86bfeff70"
server_key_iv: "0cad4f6ff30909d0c588f53fb3b3e2eb"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.579896 27551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "68811b9ab94f4d10ad1a723f1f425f50"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "c48be6b4af034cd336951cd86bfeff70"
server_key_iv: "0cad4f6ff30909d0c588f53fb3b3e2eb"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.583691 27551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 10:52:28.586206 27566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.587440 27551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:52:28.587577 27551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "68811b9ab94f4d10ad1a723f1f425f50"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "c48be6b4af034cd336951cd86bfeff70"
server_key_iv: "0cad4f6ff30909d0c588f53fb3b3e2eb"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.587687 27551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:28.605044 27551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:28.609233 27551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:28.609606 27551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:28.610399 27551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:28.611419 27551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:28.611488 27551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.611562 27551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:28.611599 27551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.620934 27551 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:41629
I20260927 10:52:28.620955 27679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:41629 every 8 connection(s)
I20260927 10:52:28.621840 27551 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:28.629980 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27551
I20260927 10:52:28.630144 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:52:28.630448 9374 external_mini_cluster.cc:1428] Setting key eea1cc9e852966f91cbf36f241d4d55a
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:28.636628 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:28.623768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:57123 (local address 127.9.39.190:36751)
0927 10:52:28.623922 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:28.623927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:28.624853 (+ 926us) server_negotiation.cc:407] Connection header received
0927 10:52:28.626200 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:28.626213 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:28.626280 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:28.626367 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:28.627854 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.628411 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.629574 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.629820 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.629946 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:28.632705 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:28.632708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:28.632711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:28.632745 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:28.634147 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:28.634805 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:28.634808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:28.634809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:28.634888 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:28.635305 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:28.635307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:28.635309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:28.635487 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:28.635563 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:28.636340 (+ 777us) server_negotiation.cc:299] Negotiation successful
0927 10:52:28.636470 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 10:52:28.637858 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:28.624199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36751 (local address 127.9.39.129:57123)
0927 10:52:28.624697 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:28.624742 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:28.625929 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:28.626573 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:28.626580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:28.627071 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:28.627681 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:28.627695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.628532 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.628535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:28.629465 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:28.629474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.629993 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.629997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:28.630253 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:28.630915 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:28.630938 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:28.632534 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:28.634283 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:28.634289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:28.634305 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:28.634677 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:28.634981 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:28.634987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:28.634992 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:28.635202 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:28.635594 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:28.635604 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:28.635971 (+ 367us) client_negotiation.cc:769] Sending connection context
0927 10:52:28.636217 (+ 246us) client_negotiation.cc:240] Negotiation successful
0927 10:52:28.636612 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":116,"threads_started":1}
I20260927 10:52:28.639204 27680 heartbeater.cc:347] Connected to a master server at 127.9.39.190:36751
I20260927 10:52:28.639461 27680 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:28.639966 27680 heartbeater.cc:511] Master 127.9.39.190:36751 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:52:28.641595 27489 ts_manager.cc:194] Registered new tserver with Master: 68811b9ab94f4d10ad1a723f1f425f50 (127.9.39.129:41629)
I20260927 10:52:28.642827 27489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:57123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:52:28.660728 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36751
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:28.774045 27685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:28.774360 27685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:28.774442 27685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:28.778580 27685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:28.778680 27685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:28.778800 27685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:28.784468 27685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:28.785555 27685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:28.786573 27685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:28.795982 27691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:28.796325 27685 server_base.cc:1034] running on GCE node
W20260927 10:52:28.795977 27693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:28.795982 27690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:28.796804 27685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:28.797433 27685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:28.798625 27685 hybrid_clock.cc:648] HybridClock initialized: now 1790506348798598 us; error 36 us; skew 500 ppm
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:28.801604 27685 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:52:28.802407 27685 webserver.cc:460] Webserver started at http://127.9.39.130:34991/ using document root <none> and password file <none>
I20260927 10:52:28.802989 27685 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:28.803049 27685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:28.803259 27685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:28.805264 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "6020a9c532324fea9380e11b1741b9f4"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "9ed90e21bb707c58a0466ab129073d34"
server_key_iv: "6c39239496b8022497648dc7e5d99212"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.805804 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "6020a9c532324fea9380e11b1741b9f4"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "9ed90e21bb707c58a0466ab129073d34"
server_key_iv: "6c39239496b8022497648dc7e5d99212"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.809486 27685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 10:52:28.811874 27700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.813300 27685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:28.813436 27685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "6020a9c532324fea9380e11b1741b9f4"
format_stamp: "Formatted at 2026-09-27 10:52:28 on dist-test-slave-p9q6"
server_key: "9ed90e21bb707c58a0466ab129073d34"
server_key_iv: "6c39239496b8022497648dc7e5d99212"
server_key_version: "encryptionkey@0"
I20260927 10:52:28.813566 27685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:28.823539 27685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:28.826812 27685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:28.826994 27685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:28.827548 27685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:28.828337 27685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:28.828404 27685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.828469 27685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:28.828517 27685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:28.836803 27685 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:34665
I20260927 10:52:28.836850 27813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:34665 every 8 connection(s)
I20260927 10:52:28.837697 27685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
Sep 27 10:52:28 dist-test-slave-p9q6 krb5kdc[27444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:28.847178 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27685
I20260927 10:52:28.847318 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:52:28.847661 9374 external_mini_cluster.cc:1428] Setting key b4f3240b915a56728a6c409b032d171e
I20260927 10:52:28.850979 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:28.839515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:37659 (local address 127.9.39.190:36751)
0927 10:52:28.839675 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:28.839679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:28.840457 (+ 778us) server_negotiation.cc:407] Connection header received
0927 10:52:28.841643 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:28.841646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:28.841693 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:28.841766 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:28.842932 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.843441 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.844504 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.844709 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.844783 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:28.847173 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:28.847176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:28.847180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:28.847208 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:28.848791 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:28.849350 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:28.849353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:28.849354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:28.849425 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:28.849786 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:28.849788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:28.849789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:28.849943 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:28.850015 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:28.850619 (+ 604us) server_negotiation.cc:299] Negotiation successful
0927 10:52:28.850776 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 10:52:28.851845 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:28.839881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36751 (local address 127.9.39.130:37659)
0927 10:52:28.840317 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:28.840358 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:28.841463 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:28.841922 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:28.841929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:28.842269 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:28.842807 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:28.842823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.843534 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.843536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:28.844408 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:28.844416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:28.844827 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:28.844830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:28.844964 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:28.845542 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:28.845558 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:28.847023 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:28.848899 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:28.848905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:28.848918 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:28.849259 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:28.849511 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:28.849515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:28.849517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:28.849698 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:28.850037 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:28.850046 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:28.850358 (+ 312us) client_negotiation.cc:769] Sending connection context
0927 10:52:28.850578 (+ 220us) client_negotiation.cc:240] Negotiation successful
0927 10:52:28.850954 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
I20260927 10:52:28.852919 27814 heartbeater.cc:347] Connected to a master server at 127.9.39.190:36751
I20260927 10:52:28.853176 27814 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:28.853645 27814 heartbeater.cc:511] Master 127.9.39.190:36751 requested a full tablet report, sending...
I20260927 10:52:28.854597 27489 ts_manager.cc:194] Registered new tserver with Master: 6020a9c532324fea9380e11b1741b9f4 (127.9.39.130:34665)
I20260927 10:52:28.855157 27489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:37659
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:52:28.874491 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36751
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:28.984915 27819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:28.985239 27819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:28.985320 27819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:28.989377 27819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:28.989473 27819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:28.989588 27819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:28.995191 27819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:28.996301 27819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:28.997341 27819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:29.007077 27824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:29.007120 27819 server_base.cc:1034] running on GCE node
W20260927 10:52:29.007084 27827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:29.007084 27825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:29.007726 27819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:29.008384 27819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:29.009543 27819 hybrid_clock.cc:648] HybridClock initialized: now 1790506349009512 us; error 48 us; skew 500 ppm
Sep 27 10:52:29 dist-test-slave-p9q6 krb5kdc[27444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:29.012369 27819 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:52:29.013306 27819 webserver.cc:460] Webserver started at http://127.9.39.131:41843/ using document root <none> and password file <none>
I20260927 10:52:29.013897 27819 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:29.013964 27819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:29.014178 27819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:29.016422 27819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "f2c4d2f1aff040168d43199151179f45"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "689a13d5f34c62a3f3206b29db4d54ed"
server_key_iv: "a1b9ad4a6cd7472ff2181ea8cc157024"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.016980 27819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "f2c4d2f1aff040168d43199151179f45"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "689a13d5f34c62a3f3206b29db4d54ed"
server_key_iv: "a1b9ad4a6cd7472ff2181ea8cc157024"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.020790 27819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 10:52:29.023227 27834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.024435 27819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:29.024554 27819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "f2c4d2f1aff040168d43199151179f45"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "689a13d5f34c62a3f3206b29db4d54ed"
server_key_iv: "a1b9ad4a6cd7472ff2181ea8cc157024"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.024658 27819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:29.042295 27819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:29.045739 27819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:29.045943 27819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:29.046561 27819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:29.047694 27819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:29.047746 27819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.047820 27819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:29.047858 27819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.057044 27819 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:40469
I20260927 10:52:29.057075 27947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:40469 every 8 connection(s)
I20260927 10:52:29.057971 27819 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:29.060688 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27819
I20260927 10:52:29.060859 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:52:29.061157 9374 external_mini_cluster.cc:1428] Setting key 42b039ffd9664889d90a4103f1677ec7
Sep 27 10:52:29 dist-test-slave-p9q6 krb5kdc[27444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:29.072268 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.059781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:53739 (local address 127.9.39.190:36751)
0927 10:52:29.059924 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.059928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.060828 (+ 900us) server_negotiation.cc:407] Connection header received
0927 10:52:29.062219 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.062223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.062278 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.062351 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:29.063670 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.064208 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.065614 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.065927 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.066041 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.068694 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:29.068697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:29.068699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:29.068725 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:29.070296 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:29.070855 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:29.070858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:29.070859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:29.070937 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:29.071292 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:29.071295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:29.071296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:29.071489 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:29.071567 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.072033 (+ 466us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.072137 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:52:29.073078 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.060217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:36751 (local address 127.9.39.131:53739)
0927 10:52:29.060681 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.060734 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.062021 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.062545 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.062552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.062973 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:29.063545 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.063559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.064338 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.064341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.065472 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.065481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.066155 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.066159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.066307 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.067009 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:29.067026 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:29.068524 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:29.070422 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:29.070427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:29.070439 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:29.070754 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:29.071044 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:29.071046 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:29.071047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:29.071187 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:29.071573 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:29.071578 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:29.071811 (+ 233us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.071960 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.072227 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":132,"threads_started":1}
I20260927 10:52:29.074366 27948 heartbeater.cc:347] Connected to a master server at 127.9.39.190:36751
I20260927 10:52:29.074656 27948 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:29.075289 27948 heartbeater.cc:511] Master 127.9.39.190:36751 requested a full tablet report, sending...
I20260927 10:52:29.076225 27489 ts_manager.cc:194] Registered new tserver with Master: f2c4d2f1aff040168d43199151179f45 (127.9.39.131:40469)
I20260927 10:52:29.076792 27489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:53739
I20260927 10:52:29.087126 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:52:29.097117 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.089108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.9.39.190:36751)
0927 10:52:29.089241 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.089246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.089418 (+ 172us) server_negotiation.cc:407] Connection header received
0927 10:52:29.089605 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.089608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.089659 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.089739 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:29.090432 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.091043 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.091931 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.092154 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.092281 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.093865 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:29.093869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:29.093872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:29.093904 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:29.095262 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:29.095788 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:29.095796 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:29.095798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:29.095875 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:29.096194 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:29.096197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:29.096199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:29.096414 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:29.096571 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.096832 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.096961 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:52:29.099999 27489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:52:29.103003 27489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:52:29.117348 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.111676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:40469 (local address 127.0.0.1:43244)
0927 10:52:29.112219 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.112230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.112311 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.113038 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.113044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.113066 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:29.113189 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.113196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.114728 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.114732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.115918 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.115928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.116974 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.117070 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.117098 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.117167 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.117234 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":60,"thread_start_us":92,"threads_started":1}
I20260927 10:52:29.117447 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.111146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51400 (local address 127.9.39.130:34665)
0927 10:52:29.111757 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.111769 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.111809 (+ 40us) server_negotiation.cc:407] Connection header received
0927 10:52:29.112037 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.112043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.112614 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.112771 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:29.113165 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.114065 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.115750 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.116610 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116720 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.116866 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.117242 (+ 376us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.117326 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":107,"threads_started":1}
I20260927 10:52:29.117443 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.112193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.9.39.129:41629)
0927 10:52:29.112538 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.112556 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.112710 (+ 154us) server_negotiation.cc:407] Connection header received
0927 10:52:29.112783 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.112788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.112959 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.113222 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:29.113370 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.114484 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.115996 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.116637 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116755 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.116892 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.117188 (+ 296us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.117286 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":67,"threads_started":1}
I20260927 10:52:29.117348 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.111157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:41629 (local address 127.0.0.1:54332)
0927 10:52:29.111925 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.111938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.112031 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.113117 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.113120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.113132 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:29.113268 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.113275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.114616 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.114619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.115875 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.115882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.116755 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.116904 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.116954 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.117056 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.117184 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":80,"thread_start_us":111,"threads_started":1}
I20260927 10:52:29.118156 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.112146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43244 (local address 127.9.39.131:40469)
0927 10:52:29.112457 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.112468 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.112488 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:52:29.112560 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.112565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.112819 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.113000 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:29.113317 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.114544 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116053 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.116875 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116978 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.117096 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.117898 (+ 802us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.118012 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":107,"threads_started":1}
I20260927 10:52:29.118464 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.111006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:34665 (local address 127.0.0.1:51400)
0927 10:52:29.111641 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.111689 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.111816 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.112824 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.112832 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.112854 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:29.113031 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.113041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.114189 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.114196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.115630 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.115640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.116934 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.116939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.117027 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.117050 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.117133 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.118336 (+ 1203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":1121,"thread_start_us":73,"threads_started":1}
I20260927 10:52:29.120529 27614 tablet_service.cc:1467] Processing CreateTablet for tablet bc6543a356a54f518c7fca1f3eb44fc1 (DEFAULT_TABLE table=test-table [id=06814d1bd9d84ff399861d0c490f3fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:29.120872 27882 tablet_service.cc:1467] Processing CreateTablet for tablet bc6543a356a54f518c7fca1f3eb44fc1 (DEFAULT_TABLE table=test-table [id=06814d1bd9d84ff399861d0c490f3fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:29.120908 27748 tablet_service.cc:1467] Processing CreateTablet for tablet bc6543a356a54f518c7fca1f3eb44fc1 (DEFAULT_TABLE table=test-table [id=06814d1bd9d84ff399861d0c490f3fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:29.121591 27614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc6543a356a54f518c7fca1f3eb44fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:29.121871 27882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc6543a356a54f518c7fca1f3eb44fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:29.121865 27748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc6543a356a54f518c7fca1f3eb44fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:29.128463 27965 tablet_bootstrap.cc:492] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50: Bootstrap starting.
I20260927 10:52:29.129686 27966 tablet_bootstrap.cc:492] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Bootstrap starting.
I20260927 10:52:29.129884 27967 tablet_bootstrap.cc:492] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Bootstrap starting.
I20260927 10:52:29.130834 27965 tablet_bootstrap.cc:654] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:29.131861 27965 log.cc:824] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:29.132247 27967 tablet_bootstrap.cc:654] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:29.132247 27966 tablet_bootstrap.cc:654] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:29.133090 27966 log.cc:824] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:29.133083 27967 log.cc:824] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:29.134173 27965 tablet_bootstrap.cc:492] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50: No bootstrap required, opened a new log
I20260927 10:52:29.134389 27965 ts_tablet_manager.cc:1397] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260927 10:52:29.135012 27967 tablet_bootstrap.cc:492] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: No bootstrap required, opened a new log
I20260927 10:52:29.135013 27966 tablet_bootstrap.cc:492] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: No bootstrap required, opened a new log
I20260927 10:52:29.135190 27967 ts_tablet_manager.cc:1397] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260927 10:52:29.135190 27966 ts_tablet_manager.cc:1397] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:52:29.137337 27965 raft_consensus.cc:348] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.137575 27965 raft_consensus.cc:374] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:29.137627 27965 raft_consensus.cc:729] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68811b9ab94f4d10ad1a723f1f425f50, State: Initialized, Role: FOLLOWER
I20260927 10:52:29.138062 27965 consensus_queue.cc:260] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.138737 27680 heartbeater.cc:503] Master 127.9.39.190:36751 was elected leader, sending a full tablet report...
I20260927 10:52:29.139107 27965 ts_tablet_manager.cc:1428] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260927 10:52:29.138953 27967 raft_consensus.cc:348] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.139216 27967 raft_consensus.cc:374] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:29.139067 27966 raft_consensus.cc:348] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.139286 27967 raft_consensus.cc:729] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6020a9c532324fea9380e11b1741b9f4, State: Initialized, Role: FOLLOWER
I20260927 10:52:29.139307 27966 raft_consensus.cc:374] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:29.139353 27966 raft_consensus.cc:729] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c4d2f1aff040168d43199151179f45, State: Initialized, Role: FOLLOWER
I20260927 10:52:29.139741 27967 consensus_queue.cc:260] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.139741 27966 consensus_queue.cc:260] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.140651 27966 ts_tablet_manager.cc:1428] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:52:29.140681 27967 ts_tablet_manager.cc:1428] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 10:52:29.141054 27948 heartbeater.cc:503] Master 127.9.39.190:36751 was elected leader, sending a full tablet report...
I20260927 10:52:29.141183 27814 heartbeater.cc:503] Master 127.9.39.190:36751 was elected leader, sending a full tablet report...
I20260927 10:52:29.147948 27972 raft_consensus.cc:482] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:29.148109 27972 raft_consensus.cc:504] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.149319 27972 leader_election.cc:288] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6020a9c532324fea9380e11b1741b9f4 (127.9.39.130:34665), 68811b9ab94f4d10ad1a723f1f425f50 (127.9.39.129:41629)
I20260927 10:52:29.153949 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.149477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:34665 (local address 127.9.39.131:41381)
0927 10:52:29.149575 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.149591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.149674 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.150019 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.150022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.150041 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:29.150169 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.150175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.151228 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.151230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.152586 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.152591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.153465 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.153468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.153674 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.153706 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.153769 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.153818 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":100}
I20260927 10:52:29.153966 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.149604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:54883 (local address 127.9.39.129:41629)
0927 10:52:29.149722 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.149729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.150153 (+ 424us) server_negotiation.cc:407] Connection header received
0927 10:52:29.150257 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.150259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.150306 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.150385 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:29.150581 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.151299 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.152714 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.153336 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.153453 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.153507 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.153760 (+ 253us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.153839 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 10:52:29.153967 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.149670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:41381 (local address 127.9.39.130:34665)
0927 10:52:29.149772 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.149777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.149794 (+ 17us) server_negotiation.cc:407] Connection header received
0927 10:52:29.149849 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.149856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.149910 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.150000 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:29.150274 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.151108 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.152714 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.153336 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.153453 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.153506 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.153792 (+ 286us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.153855 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 10:52:29.153964 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.149477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:41629 (local address 127.9.39.131:54883)
0927 10:52:29.150046 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.150060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.150162 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.150390 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.150391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.150403 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:29.150489 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.150492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.151397 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.151400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.152579 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.152590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.153465 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.153468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.153590 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.153638 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.153712 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.153797 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":295,"thread_start_us":127,"threads_started":1}
I20260927 10:52:29.154573 27634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc6543a356a54f518c7fca1f3eb44fc1" candidate_uuid: "f2c4d2f1aff040168d43199151179f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68811b9ab94f4d10ad1a723f1f425f50" is_pre_election: true
I20260927 10:52:29.154772 27768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc6543a356a54f518c7fca1f3eb44fc1" candidate_uuid: "f2c4d2f1aff040168d43199151179f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6020a9c532324fea9380e11b1741b9f4" is_pre_election: true
I20260927 10:52:29.154887 27634 raft_consensus.cc:2454] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c4d2f1aff040168d43199151179f45 in term 0.
I20260927 10:52:29.155055 27768 raft_consensus.cc:2454] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c4d2f1aff040168d43199151179f45 in term 0.
I20260927 10:52:29.155371 27835 leader_election.cc:302] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68811b9ab94f4d10ad1a723f1f425f50, f2c4d2f1aff040168d43199151179f45; no voters:
I20260927 10:52:29.155632 27972 raft_consensus.cc:2781] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:52:29.155706 27972 raft_consensus.cc:482] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:52:29.155736 27972 raft_consensus.cc:3037] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:29.156759 27972 raft_consensus.cc:504] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.157100 27972 leader_election.cc:288] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [CANDIDATE]: Term 1 election: Requested vote from peers 6020a9c532324fea9380e11b1741b9f4 (127.9.39.130:34665), 68811b9ab94f4d10ad1a723f1f425f50 (127.9.39.129:41629)
I20260927 10:52:29.157435 27768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc6543a356a54f518c7fca1f3eb44fc1" candidate_uuid: "f2c4d2f1aff040168d43199151179f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6020a9c532324fea9380e11b1741b9f4"
I20260927 10:52:29.157572 27768 raft_consensus.cc:3037] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:29.157624 27634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc6543a356a54f518c7fca1f3eb44fc1" candidate_uuid: "f2c4d2f1aff040168d43199151179f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68811b9ab94f4d10ad1a723f1f425f50"
I20260927 10:52:29.157753 27634 raft_consensus.cc:3037] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:29.158634 27768 raft_consensus.cc:2454] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c4d2f1aff040168d43199151179f45 in term 1.
I20260927 10:52:29.158864 27634 raft_consensus.cc:2454] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c4d2f1aff040168d43199151179f45 in term 1.
I20260927 10:52:29.159006 27836 leader_election.cc:302] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6020a9c532324fea9380e11b1741b9f4, f2c4d2f1aff040168d43199151179f45; no voters:
I20260927 10:52:29.159176 27972 raft_consensus.cc:2781] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:29.159403 27972 raft_consensus.cc:686] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 1 LEADER]: Becoming Leader. State: Replica: f2c4d2f1aff040168d43199151179f45, State: Running, Role: LEADER
I20260927 10:52:29.159725 27972 consensus_queue.cc:237] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } }
I20260927 10:52:29.162851 27489 catalog_manager.cc:5495] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c4d2f1aff040168d43199151179f45 (127.9.39.131). New cstate: current_term: 1 leader_uuid: "f2c4d2f1aff040168d43199151179f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2c4d2f1aff040168d43199151179f45" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 40469 } health_report { overall_health: HEALTHY } } }
I20260927 10:52:29.180478 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.176815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.9.39.131:40469)
0927 10:52:29.176994 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.177003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.177016 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:52:29.177159 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.177161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.177215 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.177304 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:52:29.177538 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.178020 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.178948 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.179163 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.179241 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.179646 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:52:29.180061 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:52:29.180191 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.180318 (+ 127us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.180352 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:52:29.186338 27634 raft_consensus.cc:1261] T bc6543a356a54f518c7fca1f3eb44fc1 P 68811b9ab94f4d10ad1a723f1f425f50 [term 1 FOLLOWER]: Refusing update from remote peer f2c4d2f1aff040168d43199151179f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:29.186362 27768 raft_consensus.cc:1261] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 1 FOLLOWER]: Refusing update from remote peer f2c4d2f1aff040168d43199151179f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:29.187034 27972 consensus_queue.cc:1035] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68811b9ab94f4d10ad1a723f1f425f50" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 41629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:29.187218 27975 consensus_queue.cc:1035] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6020a9c532324fea9380e11b1741b9f4" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 34665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:29.194929 27979 mvcc.cc:205] Tried to move back new op lower bound from 7333914006260322304 to 7333914006161010688. Current Snapshot: MvccSnapshot[applied={T|T < 7333914006260322304}]
I20260927 10:52:29.203521 27489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:29.203703 27489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:29.206151 27489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e3826829e264e96bd59ac526c7dee84: Sending DeleteTablet for 3 replicas of tablet bc6543a356a54f518c7fca1f3eb44fc1
I20260927 10:52:29.206967 27748 tablet_service.cc:1514] Processing DeleteTablet for tablet bc6543a356a54f518c7fca1f3eb44fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:29 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:51400
I20260927 10:52:29.207108 27614 tablet_service.cc:1514] Processing DeleteTablet for tablet bc6543a356a54f518c7fca1f3eb44fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:29 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:54332
I20260927 10:52:29.207546 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27551
I20260927 10:52:29.207837 27882 tablet_service.cc:1514] Processing DeleteTablet for tablet bc6543a356a54f518c7fca1f3eb44fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:29 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:43244
I20260927 10:52:29.208297 27989 tablet_replica.cc:330] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: stopping tablet replica
I20260927 10:52:29.208636 27989 raft_consensus.cc:2229] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:52:29.209028 27989 raft_consensus.cc:2258] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:29.209431 27987 tablet_replica.cc:330] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: stopping tablet replica
I20260927 10:52:29.209743 27987 raft_consensus.cc:2229] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:29.210058 27987 raft_consensus.cc:2258] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:29.211206 27989 ts_tablet_manager.cc:1905] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:29.211366 27987 ts_tablet_manager.cc:1905] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:29.214047 27987 ts_tablet_manager.cc:1918] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:29.214054 27989 ts_tablet_manager.cc:1918] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:29.214155 27987 log.cc:1196] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/bc6543a356a54f518c7fca1f3eb44fc1
I20260927 10:52:29.214339 27989 log.cc:1196] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/bc6543a356a54f518c7fca1f3eb44fc1
I20260927 10:52:29.214493 27987 ts_tablet_manager.cc:1939] T bc6543a356a54f518c7fca1f3eb44fc1 P 6020a9c532324fea9380e11b1741b9f4: Deleting consensus metadata
I20260927 10:52:29.214828 27989 ts_tablet_manager.cc:1939] T bc6543a356a54f518c7fca1f3eb44fc1 P f2c4d2f1aff040168d43199151179f45: Deleting consensus metadata
I20260927 10:52:29.215839 27474 catalog_manager.cc:4841] TS 6020a9c532324fea9380e11b1741b9f4 (127.9.39.130:34665): tablet bc6543a356a54f518c7fca1f3eb44fc1 (table test-table [id=06814d1bd9d84ff399861d0c490f3fce]) successfully deleted
I20260927 10:52:29.215848 27473 catalog_manager.cc:4841] TS f2c4d2f1aff040168d43199151179f45 (127.9.39.131:40469): tablet bc6543a356a54f518c7fca1f3eb44fc1 (table test-table [id=06814d1bd9d84ff399861d0c490f3fce]) successfully deleted
W20260927 10:52:29.218035 27473 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:41629 (error 108)
I20260927 10:52:29.218230 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27685
I20260927 10:52:29.219440 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.219188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:41629 (local address 127.0.0.1:54340)
0927 10:52:29.219307 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.219372 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:41629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 10:52:29.219640 27473 catalog_manager.cc:4584] TS 68811b9ab94f4d10ad1a723f1f425f50 (127.9.39.129:41629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc6543a356a54f518c7fca1f3eb44fc1: Network error: Client connection negotiation failed: client connection to 127.9.39.129:41629: connect: Connection refused (error 111)
I20260927 10:52:29.226769 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27819
I20260927 10:52:29.232373 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27457
2026-09-27T10:52:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2268 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T10:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:29Z Disabled control of system clock
I20260927 10:52:29.262430 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:35071
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:35071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 10:52:29.371730 27994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:29.371979 27994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:29.372015 27994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:29.375900 27994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:29.375949 27994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:29.375964 27994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:29.375981 27994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:29.375996 27994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:29.380722 27994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:35071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:35071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:29.381688 27994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:29.382990 27994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:52:29.391258 27994 server_base.cc:1034] running on GCE node
W20260927 10:52:29.391218 27999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:29.391223 28002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:29.391193 28000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:29.392134 27994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:29.393254 27994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:29.394438 27994 hybrid_clock.cc:648] HybridClock initialized: now 1790506349394404 us; error 44 us; skew 500 ppm
I20260927 10:52:29.396339 27994 webserver.cc:460] Webserver started at http://127.9.39.190:37823/ using document root <none> and password file <none>
I20260927 10:52:29.396930 27994 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:29.396992 27994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:29.397234 27994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:29.399339 27994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "8f96ca4ed4674900a9267b03e03d258c"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "086bc6f4df6438cf409ab15862cd533b"
server_key_iv: "c41abf2c0cb3da6adf8f6b5cda6183ee"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.399915 27994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "8f96ca4ed4674900a9267b03e03d258c"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "086bc6f4df6438cf409ab15862cd533b"
server_key_iv: "c41abf2c0cb3da6adf8f6b5cda6183ee"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.403672 27994 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:52:29.406069 28008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.407323 27994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:52:29.407459 27994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "8f96ca4ed4674900a9267b03e03d258c"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "086bc6f4df6438cf409ab15862cd533b"
server_key_iv: "c41abf2c0cb3da6adf8f6b5cda6183ee"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.407581 27994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:29.422350 27994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:29.423095 27994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:29.423296 27994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:29.431114 27994 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:35071
I20260927 10:52:29.431162 28060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:35071 every 8 connection(s)
I20260927 10:52:29.432049 27994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:29.435071 28061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:29.438102 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 27994
I20260927 10:52:29.438243 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:29.438532 9374 external_mini_cluster.cc:1428] Setting key 2241ecdef54e12e56ab09b7248e77911
I20260927 10:52:29.440956 28061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: Bootstrap starting.
I20260927 10:52:29.443132 28061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:29.443846 28061 log.cc:824] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:29.445739 28061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: No bootstrap required, opened a new log
I20260927 10:52:29.447537 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.439939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.9.39.190:35071)
0927 10:52:29.440588 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.440603 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.440638 (+ 35us) server_negotiation.cc:407] Connection header received
0927 10:52:29.441305 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.441324 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.441719 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.442038 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:29.442468 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.443517 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.444432 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.444889 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.445054 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.445879 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:29.445884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:29.445896 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:29.446269 (+ 373us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:29.446323 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:29.446341 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:29.446444 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.446589 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.446867 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":127,"threads_started":1}
I20260927 10:52:29.449311 28061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f96ca4ed4674900a9267b03e03d258c" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35071 } }
I20260927 10:52:29.449579 28061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:29.449633 28061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f96ca4ed4674900a9267b03e03d258c, State: Initialized, Role: FOLLOWER
I20260927 10:52:29.449990 28061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f96ca4ed4674900a9267b03e03d258c" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35071 } }
I20260927 10:52:29.450095 28061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:29.450137 28061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:29.450208 28061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:29.451078 28061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f96ca4ed4674900a9267b03e03d258c" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35071 } }
I20260927 10:52:29.451360 28061 leader_election.cc:302] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f96ca4ed4674900a9267b03e03d258c; no voters:
I20260927 10:52:29.451592 28061 leader_election.cc:288] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:29.451709 28066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:29.451946 28066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [term 1 LEADER]: Becoming Leader. State: Replica: 8f96ca4ed4674900a9267b03e03d258c, State: Running, Role: LEADER
I20260927 10:52:29.452250 28066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f96ca4ed4674900a9267b03e03d258c" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35071 } }
I20260927 10:52:29.452736 28061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:29.453764 28068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f96ca4ed4674900a9267b03e03d258c. Latest consensus state: current_term: 1 leader_uuid: "8f96ca4ed4674900a9267b03e03d258c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f96ca4ed4674900a9267b03e03d258c" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35071 } } }
I20260927 10:52:29.453887 28068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:29.455708 28067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f96ca4ed4674900a9267b03e03d258c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f96ca4ed4674900a9267b03e03d258c" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35071 } } }
I20260927 10:52:29.455955 28067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c [sys.catalog]: This master's current role is: LEADER
W20260927 10:52:29.458297 28081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:52:29.458376 28081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:52:29.458453 28075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:29.459198 28075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:29.464766 28075 catalog_manager.cc:1347] Generated new cluster ID: 00d5a88d63854b53ba7dd8753e7a840d
I20260927 10:52:29.464828 28075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:29.500119 28075 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:29.501386 28075 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:29.507715 28075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: Generated new TSK 0
I20260927 10:52:29.508425 28075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:29.523085 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35071
--builtin_ntp_servers=127.9.39.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 10:52:29.632906 28085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:29.633174 28085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:29.633213 28085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:29.637200 28085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:29.637259 28085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:29.637312 28085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:29.642077 28085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:29.643111 28085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:29.644404 28085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:29.653868 28091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:29.653869 28093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:29.653967 28085 server_base.cc:1034] running on GCE node
W20260927 10:52:29.654206 28090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:29.654646 28085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:29.655380 28085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:29.656559 28085 hybrid_clock.cc:648] HybridClock initialized: now 1790506349656547 us; error 38 us; skew 500 ppm
I20260927 10:52:29.658481 28085 webserver.cc:460] Webserver started at http://127.9.39.129:33719/ using document root <none> and password file <none>
I20260927 10:52:29.659123 28085 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:29.659210 28085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:29.659418 28085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:29.661463 28085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "a86cb1f3be084d23989b2eec78c00db1"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "2811124f5eb42d38295cf8ba41e9ba1f"
server_key_iv: "2c499b2e5f1a5b2ae89f6085d01fc79c"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.662031 28085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "a86cb1f3be084d23989b2eec78c00db1"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "2811124f5eb42d38295cf8ba41e9ba1f"
server_key_iv: "2c499b2e5f1a5b2ae89f6085d01fc79c"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.666055 28085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 10:52:29.668565 28099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.669787 28085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 10:52:29.669920 28085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "a86cb1f3be084d23989b2eec78c00db1"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "2811124f5eb42d38295cf8ba41e9ba1f"
server_key_iv: "2c499b2e5f1a5b2ae89f6085d01fc79c"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.670033 28085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:29.686039 28085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:29.686779 28085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:29.686972 28085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:29.687561 28085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:29.688512 28085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:29.688580 28085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.688649 28085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:29.688690 28085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.697575 28085 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:39451
I20260927 10:52:29.697605 28212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:39451 every 8 connection(s)
I20260927 10:52:29.698475 28085 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:29.699517 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28085
I20260927 10:52:29.699682 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:52:29.699983 9374 external_mini_cluster.cc:1428] Setting key 023b3865749e07120376d2906bc39035
I20260927 10:52:29.701972 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35071
--builtin_ntp_servers=127.9.39.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 10:52:29.709017 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.700320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:38455 (local address 127.9.39.190:35071)
0927 10:52:29.700471 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.700476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.701794 (+ 1318us) server_negotiation.cc:407] Connection header received
0927 10:52:29.703424 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.703427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.703471 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.703532 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:29.704604 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.705123 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.706968 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.707189 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.707269 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.708565 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:29.708569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:29.708572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:29.708585 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:29.708605 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:29.708613 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:29.708670 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.708857 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.708906 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:52:29.710304 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.700853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:35071 (local address 127.9.39.129:38455)
0927 10:52:29.701661 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.701701 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.703175 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.703748 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.703760 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.703873 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:29.704487 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.704500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.705253 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.705259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.706850 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.706863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.707353 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.707359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.707598 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.708249 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:29.708252 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:29.708319 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:29.708409 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:29.708698 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:29.708710 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:29.708718 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.708863 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.709193 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":204,"threads_started":1}
I20260927 10:52:29.711670 28213 heartbeater.cc:347] Connected to a master server at 127.9.39.190:35071
I20260927 10:52:29.711906 28213 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:29.712412 28213 heartbeater.cc:511] Master 127.9.39.190:35071 requested a full tablet report, sending...
I20260927 10:52:29.713944 28025 ts_manager.cc:194] Registered new tserver with Master: a86cb1f3be084d23989b2eec78c00db1 (127.9.39.129:39451)
I20260927 10:52:29.715497 28025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.129:38455
W20260927 10:52:29.816095 28216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:29.816352 28216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:29.816390 28216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:29.820276 28216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:29.820326 28216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:29.820379 28216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:29.824998 28216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:29.826000 28216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:29.827288 28216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:29.836678 28221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:29.836683 28222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:29.836683 28224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:29.836944 28216 server_base.cc:1034] running on GCE node
I20260927 10:52:29.837373 28216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:29.837984 28216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:29.839144 28216 hybrid_clock.cc:648] HybridClock initialized: now 1790506349839121 us; error 28 us; skew 500 ppm
I20260927 10:52:29.840917 28216 webserver.cc:460] Webserver started at http://127.9.39.130:37221/ using document root <none> and password file <none>
I20260927 10:52:29.841478 28216 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:29.841538 28216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:29.841761 28216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:29.843783 28216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "afc023fbe417b9d336e9bc7c1ed30f6a"
server_key_iv: "0727a1dc1c05d6c172c8b698474ec2c6"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.844336 28216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "afc023fbe417b9d336e9bc7c1ed30f6a"
server_key_iv: "0727a1dc1c05d6c172c8b698474ec2c6"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.847811 28216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:52:29.850142 28230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.851354 28216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:29.851465 28216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed"
format_stamp: "Formatted at 2026-09-27 10:52:29 on dist-test-slave-p9q6"
server_key: "afc023fbe417b9d336e9bc7c1ed30f6a"
server_key_iv: "0727a1dc1c05d6c172c8b698474ec2c6"
server_key_version: "encryptionkey@0"
I20260927 10:52:29.851564 28216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:29.876963 28216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:29.877624 28216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:29.877806 28216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:29.878407 28216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:29.879347 28216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:29.879392 28216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.879455 28216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:29.879489 28216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:29.887925 28216 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:40977
I20260927 10:52:29.887950 28343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:40977 every 8 connection(s)
I20260927 10:52:29.888832 28216 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:52:29.897850 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.890752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:41947 (local address 127.9.39.190:35071)
0927 10:52:29.890916 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:29.890920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:29.892002 (+ 1082us) server_negotiation.cc:407] Connection header received
0927 10:52:29.893367 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:29.893369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:29.893414 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:29.893487 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:29.894376 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.894980 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.896132 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.896337 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.896409 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.897445 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:29.897448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:29.897450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:29.897462 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:29.897480 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:29.897487 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:29.897542 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:29.897686 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 10:52:29.897735 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 10:52:29.898682 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28216
I20260927 10:52:29.898830 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:52:29.898856 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:29.891149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:35071 (local address 127.9.39.130:41947)
0927 10:52:29.891856 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:29.891898 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:29.893183 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:29.893680 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:29.893686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:29.893770 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:29.894254 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.894266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.895100 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.895103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.896021 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:29.896032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:29.896455 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:29.896458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:29.896621 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:29.897186 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:29.897188 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:29.897244 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:29.897307 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:29.897566 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:29.897572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:29.897576 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:52:29.897669 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 10:52:29.897899 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260927 10:52:29.899165 9374 external_mini_cluster.cc:1428] Setting key 85ea09d1ce3d93f91cc3965634f92540
I20260927 10:52:29.900022 28344 heartbeater.cc:347] Connected to a master server at 127.9.39.190:35071
I20260927 10:52:29.900262 28344 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:29.900863 28344 heartbeater.cc:511] Master 127.9.39.190:35071 requested a full tablet report, sending...
I20260927 10:52:29.901075 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35071
--builtin_ntp_servers=127.9.39.148:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 10:52:29.901862 28025 ts_manager.cc:194] Registered new tserver with Master: f59dbcb9ffdc4b55b2df9f5710d0aaed (127.9.39.130:40977)
I20260927 10:52:29.902560 28025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.130:41947
W20260927 10:52:30.008939 28347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:30.009204 28347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:30.009245 28347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:30.013686 28347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:30.013751 28347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:30.013809 28347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:30.018551 28347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:30.019702 28347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:30.021023 28347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:30.030496 28352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:30.030566 28353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:30.030681 28347 server_base.cc:1034] running on GCE node
W20260927 10:52:30.030622 28355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:30.031205 28347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:30.031803 28347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:30.032970 28347 hybrid_clock.cc:648] HybridClock initialized: now 1790506350032953 us; error 27 us; skew 500 ppm
I20260927 10:52:30.034771 28347 webserver.cc:460] Webserver started at http://127.9.39.131:43003/ using document root <none> and password file <none>
I20260927 10:52:30.035352 28347 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:30.035413 28347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:30.035640 28347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:30.037642 28347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "8354482b7f604ce2a379fc93563201b1"
format_stamp: "Formatted at 2026-09-27 10:52:30 on dist-test-slave-p9q6"
server_key: "0a6967a11217b0a540634d167b439c38"
server_key_iv: "44312262452d4fee3b8d2781216a0ac1"
server_key_version: "encryptionkey@0"
I20260927 10:52:30.038193 28347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "8354482b7f604ce2a379fc93563201b1"
format_stamp: "Formatted at 2026-09-27 10:52:30 on dist-test-slave-p9q6"
server_key: "0a6967a11217b0a540634d167b439c38"
server_key_iv: "44312262452d4fee3b8d2781216a0ac1"
server_key_version: "encryptionkey@0"
I20260927 10:52:30.041739 28347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:52:30.044183 28361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:30.045351 28347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:30.045463 28347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "8354482b7f604ce2a379fc93563201b1"
format_stamp: "Formatted at 2026-09-27 10:52:30 on dist-test-slave-p9q6"
server_key: "0a6967a11217b0a540634d167b439c38"
server_key_iv: "44312262452d4fee3b8d2781216a0ac1"
server_key_version: "encryptionkey@0"
I20260927 10:52:30.045564 28347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:30.070003 28347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:30.070765 28347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:30.070962 28347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:30.071563 28347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:30.072490 28347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:30.072535 28347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:30.072618 28347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:30.072655 28347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:30.081067 28347 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:36007
I20260927 10:52:30.081094 28474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:36007 every 8 connection(s)
I20260927 10:52:30.081923 28347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:30.087314 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28347
I20260927 10:52:30.087450 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:52:30.087739 9374 external_mini_cluster.cc:1428] Setting key 20434d8b383d9a8f6a49673c5169b612
I20260927 10:52:30.091565 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.083717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:51805 (local address 127.9.39.190:35071)
0927 10:52:30.083857 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.083860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.084893 (+ 1033us) server_negotiation.cc:407] Connection header received
0927 10:52:30.086287 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.086290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.086347 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.086452 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:30.087416 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.088098 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.089675 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.089961 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.090042 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.091171 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:30.091174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:30.091176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:30.091188 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:30.091207 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:30.091214 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:30.091273 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.091405 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.091455 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 10:52:30.092505 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.084104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:35071 (local address 127.9.39.131:51805)
0927 10:52:30.084745 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.084792 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:30.086067 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:30.086624 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:30.086630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:30.086761 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:30.087281 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.087294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.088252 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.088256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.089523 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.089535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.090098 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.090101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.090250 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.090913 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:30.090915 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:30.090959 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:30.091026 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:30.091285 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:30.091291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:30.091295 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:52:30.091392 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 10:52:30.091647 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":117,"threads_started":1}
I20260927 10:52:30.093568 28475 heartbeater.cc:347] Connected to a master server at 127.9.39.190:35071
I20260927 10:52:30.093775 28475 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:30.094255 28475 heartbeater.cc:511] Master 127.9.39.190:35071 requested a full tablet report, sending...
I20260927 10:52:30.095222 28025 ts_manager.cc:194] Registered new tserver with Master: 8354482b7f604ce2a379fc93563201b1 (127.9.39.131:36007)
I20260927 10:52:30.095791 28025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.131:51805
I20260927 10:52:30.101536 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:52:30.107146 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.103533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.9.39.190:35071)
0927 10:52:30.103673 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.103678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.103875 (+ 197us) server_negotiation.cc:407] Connection header received
0927 10:52:30.103969 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.103971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.104012 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.104080 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:30.104294 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.104796 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.105631 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.105832 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.105904 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.106654 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:30.106656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:30.106658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:30.106669 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:30.106686 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:30.106748 (+ 62us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:30.106852 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.106933 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.106965 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:52:30.109905 28025 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:52:30.113091 28025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:52:30.127259 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.121130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:36007 (local address 127.0.0.1:40732)
0927 10:52:30.121676 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.121727 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:30.121879 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:30.122655 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:30.122661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:30.122731 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:30.122869 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.122877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.124322 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.124325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.125624 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.125633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.126708 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.126814 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.126850 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 10:52:30.126972 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 10:52:30.127087 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":40,"thread_start_us":87,"threads_started":1}
I20260927 10:52:30.127285 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.121249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.9.39.131:36007)
0927 10:52:30.121735 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.121748 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.122023 (+ 275us) server_negotiation.cc:407] Connection header received
0927 10:52:30.122110 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.122115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.122323 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.122480 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:30.122999 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.124193 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.125745 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.126537 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126672 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.126882 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.127001 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.127102 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":159,"threads_started":1}
I20260927 10:52:30.127588 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.122385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.9.39.130:40977)
0927 10:52:30.123128 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.123139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.123160 (+ 21us) server_negotiation.cc:407] Connection header received
0927 10:52:30.123228 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.123233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.123401 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.123534 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:30.123745 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.124816 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126102 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.126604 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126740 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.126913 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.127319 (+ 406us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.127426 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":157,"threads_started":1}
I20260927 10:52:30.127691 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.121301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:40977 (local address 127.0.0.1:44920)
0927 10:52:30.122021 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.122035 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:30.122119 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:30.123535 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:30.123540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:30.123557 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:30.123630 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.123635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.124946 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.124950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.125996 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.126003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.126975 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.127081 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.127100 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 10:52:30.127169 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 10:52:30.127230 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260927 10:52:30.127979 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.121902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:39451 (local address 127.0.0.1:51930)
0927 10:52:30.122812 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.122823 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:30.122894 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:30.123298 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:30.123301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:30.123318 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:30.123403 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.123410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.124443 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.124446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.125755 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.125765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.126951 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.127044 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.127067 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 10:52:30.127148 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 10:52:30.127222 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":45,"thread_start_us":93,"threads_started":1}
I20260927 10:52:30.128808 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.122307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.9.39.129:39451)
0927 10:52:30.122906 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.122920 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.122947 (+ 27us) server_negotiation.cc:407] Connection header received
0927 10:52:30.123026 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.123030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.123167 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.123321 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:30.123498 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.124325 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.125874 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.126584 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.126674 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.126882 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.128537 (+ 1655us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.128657 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":136,"threads_started":1}
I20260927 10:52:30.130136 28409 tablet_service.cc:1467] Processing CreateTablet for tablet 21fbe9b5c4e04069b319101c51183159 (DEFAULT_TABLE table=test-table [id=d94d538b4d544333a0ab24be51d92825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:30.130134 28278 tablet_service.cc:1467] Processing CreateTablet for tablet 21fbe9b5c4e04069b319101c51183159 (DEFAULT_TABLE table=test-table [id=d94d538b4d544333a0ab24be51d92825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:30.131057 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fbe9b5c4e04069b319101c51183159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:30.131064 28278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fbe9b5c4e04069b319101c51183159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:30.134862 28147 tablet_service.cc:1467] Processing CreateTablet for tablet 21fbe9b5c4e04069b319101c51183159 (DEFAULT_TABLE table=test-table [id=d94d538b4d544333a0ab24be51d92825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:30.135923 28147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fbe9b5c4e04069b319101c51183159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:30.137225 28492 tablet_bootstrap.cc:492] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: Bootstrap starting.
I20260927 10:52:30.138777 28493 tablet_bootstrap.cc:492] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Bootstrap starting.
I20260927 10:52:30.139492 28492 tablet_bootstrap.cc:654] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:30.140196 28492 log.cc:824] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:30.140952 28493 tablet_bootstrap.cc:654] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:30.141749 28493 log.cc:824] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:30.141901 28492 tablet_bootstrap.cc:492] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: No bootstrap required, opened a new log
I20260927 10:52:30.142076 28492 ts_tablet_manager.cc:1397] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:52:30.142953 28496 tablet_bootstrap.cc:492] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: Bootstrap starting.
I20260927 10:52:30.143814 28493 tablet_bootstrap.cc:492] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: No bootstrap required, opened a new log
I20260927 10:52:30.144006 28493 ts_tablet_manager.cc:1397] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:52:30.145032 28496 tablet_bootstrap.cc:654] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:30.145813 28496 log.cc:824] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:30.146379 28492 raft_consensus.cc:348] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.146627 28492 raft_consensus.cc:374] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:30.146709 28492 raft_consensus.cc:729] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59dbcb9ffdc4b55b2df9f5710d0aaed, State: Initialized, Role: FOLLOWER
I20260927 10:52:30.147147 28492 consensus_queue.cc:260] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.147781 28496 tablet_bootstrap.cc:492] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: No bootstrap required, opened a new log
I20260927 10:52:30.147783 28344 heartbeater.cc:503] Master 127.9.39.190:35071 was elected leader, sending a full tablet report...
I20260927 10:52:30.147964 28496 ts_tablet_manager.cc:1397] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260927 10:52:30.147794 28493 raft_consensus.cc:348] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.148047 28493 raft_consensus.cc:374] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:30.148128 28493 raft_consensus.cc:729] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8354482b7f604ce2a379fc93563201b1, State: Initialized, Role: FOLLOWER
I20260927 10:52:30.148528 28493 consensus_queue.cc:260] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.149315 28492 ts_tablet_manager.cc:1428] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260927 10:52:30.149490 28493 ts_tablet_manager.cc:1428] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:52:30.149794 28475 heartbeater.cc:503] Master 127.9.39.190:35071 was elected leader, sending a full tablet report...
I20260927 10:52:30.153173 28496 raft_consensus.cc:348] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.153389 28496 raft_consensus.cc:374] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:30.153469 28496 raft_consensus.cc:729] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86cb1f3be084d23989b2eec78c00db1, State: Initialized, Role: FOLLOWER
I20260927 10:52:30.153900 28496 consensus_queue.cc:260] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.154592 28213 heartbeater.cc:503] Master 127.9.39.190:35071 was elected leader, sending a full tablet report...
I20260927 10:52:30.154860 28496 ts_tablet_manager.cc:1428] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260927 10:52:30.199651 28214 tablet.cc:2369] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:52:30.255365 28500 raft_consensus.cc:482] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:30.255627 28500 raft_consensus.cc:504] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.256807 28500 leader_election.cc:288] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8354482b7f604ce2a379fc93563201b1 (127.9.39.131:36007), f59dbcb9ffdc4b55b2df9f5710d0aaed (127.9.39.130:40977)
I20260927 10:52:30.260699 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.257113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:40147 (local address 127.9.39.131:36007)
0927 10:52:30.257283 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.257288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.257393 (+ 105us) server_negotiation.cc:407] Connection header received
0927 10:52:30.257531 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.257538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.257592 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.257678 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:30.257960 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.258509 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.259724 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.260128 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.260254 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.260305 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.260473 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.260554 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260927 10:52:30.260725 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.257340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:46181 (local address 127.9.39.130:40977)
0927 10:52:30.257460 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.257466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.257567 (+ 101us) server_negotiation.cc:407] Connection header received
0927 10:52:30.257689 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.257692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.257743 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.257822 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:30.258106 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.258619 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.259756 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.260134 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.260260 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.260305 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.260531 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.260621 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:52:30.260715 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.257068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:40977 (local address 127.9.39.129:46181)
0927 10:52:30.257472 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.257486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:30.257578 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:30.257842 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:30.257844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:30.257858 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:30.257999 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.258005 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.258757 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.258760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.259613 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.259622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.260271 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.260274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.260385 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.260411 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 10:52:30.260501 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 10:52:30.260562 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":39,"thread_start_us":86,"threads_started":1}
I20260927 10:52:30.260718 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.256989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:36007 (local address 127.9.39.129:40147)
0927 10:52:30.257289 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.257304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:30.257426 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:30.257696 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:30.257699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:30.257716 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:30.257847 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.257854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.258619 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.258623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.259603 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:30.259612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.260252 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.260255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:30.260345 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.260377 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 10:52:30.260437 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 10:52:30.260519 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":6,"thread_start_us":86,"threads_started":1}
I20260927 10:52:30.261679 28298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fbe9b5c4e04069b319101c51183159" candidate_uuid: "a86cb1f3be084d23989b2eec78c00db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" is_pre_election: true
I20260927 10:52:30.261708 28429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fbe9b5c4e04069b319101c51183159" candidate_uuid: "a86cb1f3be084d23989b2eec78c00db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8354482b7f604ce2a379fc93563201b1" is_pre_election: true
I20260927 10:52:30.262017 28298 raft_consensus.cc:2454] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a86cb1f3be084d23989b2eec78c00db1 in term 0.
I20260927 10:52:30.262022 28429 raft_consensus.cc:2454] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a86cb1f3be084d23989b2eec78c00db1 in term 0.
I20260927 10:52:30.262521 28101 leader_election.cc:302] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8354482b7f604ce2a379fc93563201b1, a86cb1f3be084d23989b2eec78c00db1, f59dbcb9ffdc4b55b2df9f5710d0aaed; no voters:
I20260927 10:52:30.262827 28500 raft_consensus.cc:2781] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:52:30.262913 28500 raft_consensus.cc:482] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:52:30.262962 28500 raft_consensus.cc:3037] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:30.263993 28500 raft_consensus.cc:504] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.264371 28500 leader_election.cc:288] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [CANDIDATE]: Term 1 election: Requested vote from peers 8354482b7f604ce2a379fc93563201b1 (127.9.39.131:36007), f59dbcb9ffdc4b55b2df9f5710d0aaed (127.9.39.130:40977)
I20260927 10:52:30.264714 28429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fbe9b5c4e04069b319101c51183159" candidate_uuid: "a86cb1f3be084d23989b2eec78c00db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8354482b7f604ce2a379fc93563201b1"
I20260927 10:52:30.264806 28298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fbe9b5c4e04069b319101c51183159" candidate_uuid: "a86cb1f3be084d23989b2eec78c00db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed"
I20260927 10:52:30.264832 28429 raft_consensus.cc:3037] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:30.264958 28298 raft_consensus.cc:3037] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:30.265894 28429 raft_consensus.cc:2454] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a86cb1f3be084d23989b2eec78c00db1 in term 1.
I20260927 10:52:30.266018 28298 raft_consensus.cc:2454] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a86cb1f3be084d23989b2eec78c00db1 in term 1.
I20260927 10:52:30.266268 28101 leader_election.cc:302] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8354482b7f604ce2a379fc93563201b1, a86cb1f3be084d23989b2eec78c00db1; no voters:
I20260927 10:52:30.266472 28500 raft_consensus.cc:2781] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:30.266671 28500 raft_consensus.cc:686] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 1 LEADER]: Becoming Leader. State: Replica: a86cb1f3be084d23989b2eec78c00db1, State: Running, Role: LEADER
I20260927 10:52:30.267046 28500 consensus_queue.cc:237] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } }
I20260927 10:52:30.270051 28025 catalog_manager.cc:5495] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a86cb1f3be084d23989b2eec78c00db1 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "a86cb1f3be084d23989b2eec78c00db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a86cb1f3be084d23989b2eec78c00db1" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 39451 } health_report { overall_health: HEALTHY } } }
I20260927 10:52:30.281234 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.277750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.9.39.129:39451)
0927 10:52:30.277875 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:30.277879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:30.277894 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:52:30.278021 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:30.278024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:30.278081 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:30.278173 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:52:30.278395 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.278981 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.279790 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:30.279994 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:30.280075 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:30.280305 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:52:30.280694 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:52:30.280805 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:30.281027 (+ 222us) server_negotiation.cc:299] Negotiation successful
0927 10:52:30.281071 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 10:52:30.287053 28429 raft_consensus.cc:1261] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 1 FOLLOWER]: Refusing update from remote peer a86cb1f3be084d23989b2eec78c00db1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:30.287052 28298 raft_consensus.cc:1261] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 1 FOLLOWER]: Refusing update from remote peer a86cb1f3be084d23989b2eec78c00db1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:30.287786 28503 consensus_queue.cc:1035] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8354482b7f604ce2a379fc93563201b1" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:30.287966 28500 consensus_queue.cc:1035] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59dbcb9ffdc4b55b2df9f5710d0aaed" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 40977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:30.295531 28508 mvcc.cc:205] Tried to move back new op lower bound from 7333914010768818176 to 7333914010696650752. Current Snapshot: MvccSnapshot[applied={T|T < 7333914010768818176}]
I20260927 10:52:30.304899 28024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:30.305063 28024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:30.307425 28024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f96ca4ed4674900a9267b03e03d258c: Sending DeleteTablet for 3 replicas of tablet 21fbe9b5c4e04069b319101c51183159
I20260927 10:52:30.308215 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fbe9b5c4e04069b319101c51183159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:30 UTC) from {username='slave'} at 127.0.0.1:40732
I20260927 10:52:30.308420 28147 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fbe9b5c4e04069b319101c51183159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:30 UTC) from {username='slave'} at 127.0.0.1:51930
I20260927 10:52:30.308768 28515 tablet_replica.cc:330] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: stopping tablet replica
I20260927 10:52:30.308957 28516 tablet_replica.cc:330] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1: stopping tablet replica
I20260927 10:52:30.309113 28515 raft_consensus.cc:2229] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:30.309180 28278 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fbe9b5c4e04069b319101c51183159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:30 UTC) from {username='slave'} at 127.0.0.1:44920
I20260927 10:52:30.309361 28516 raft_consensus.cc:2229] T 21fbe9b5c4e04069b319101c51183159 P a86cb1f3be084d23989b2eec78c00db1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:52:30.309443 28515 raft_consensus.cc:2258] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:30.309900 28517 tablet_replica.cc:330] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: stopping tablet replica
I20260927 10:52:30.309983 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28085
I20260927 10:52:30.310181 28517 raft_consensus.cc:2229] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:30.311071 28517 raft_consensus.cc:2258] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:30.312897 28515 ts_tablet_manager.cc:1905] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:30.315368 28515 ts_tablet_manager.cc:1918] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:30.315472 28515 log.cc:1196] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/21fbe9b5c4e04069b319101c51183159
I20260927 10:52:30.315793 28515 ts_tablet_manager.cc:1939] T 21fbe9b5c4e04069b319101c51183159 P 8354482b7f604ce2a379fc93563201b1: Deleting consensus metadata
I20260927 10:52:30.316107 28517 ts_tablet_manager.cc:1905] T 21fbe9b5c4e04069b319101c51183159 P f59dbcb9ffdc4b55b2df9f5710d0aaed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:30.316478 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28216
W20260927 10:52:30.316612 28009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:39451 (error 108)
I20260927 10:52:30.318274 28010 catalog_manager.cc:4841] TS 8354482b7f604ce2a379fc93563201b1 (127.9.39.131:36007): tablet 21fbe9b5c4e04069b319101c51183159 (table test-table [id=d94d538b4d544333a0ab24be51d92825]) successfully deleted
I20260927 10:52:30.318857 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.318495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:39451 (local address 127.0.0.1:51946)
0927 10:52:30.318645 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.318754 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 10:52:30.319008 28009 catalog_manager.cc:4584] TS a86cb1f3be084d23989b2eec78c00db1 (127.9.39.129:39451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21fbe9b5c4e04069b319101c51183159: Network error: Client connection negotiation failed: client connection to 127.9.39.129:39451: connect: Connection refused (error 111)
I20260927 10:52:30.322144 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28347
I20260927 10:52:30.323598 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:30.322256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:40977 (local address 127.0.0.1:44932)
0927 10:52:30.323431 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:30.323498 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.130:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1069}
I20260927 10:52:30.327653 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 27994
2026-09-27T10:52:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1094 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:30 dist-test-slave-p9q6 krb5kdc[28521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:30 dist-test-slave-p9q6 krb5kdc[28521](info): set up 2 sockets
Sep 27 10:52:30 dist-test-slave-p9q6 krb5kdc[28521](info): commencing operation
krb5kdc: starting...
W20260927 10:52:31.362821 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:31 dist-test-slave-p9q6 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:31.484030 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:41745
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:41745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:31.591840 28534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:31.592142 28534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:31.592216 28534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:31.596220 28534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:31.596297 28534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:31.596336 28534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:31.596393 28534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:31.596429 28534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:31.601599 28534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:41745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:31.602638 28534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:31.603650 28534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:31.611940 28539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:31.611935 28542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:31.611935 28540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:31.612048 28534 server_base.cc:1034] running on GCE node
I20260927 10:52:31.612736 28534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:31.613745 28534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:31.614910 28534 hybrid_clock.cc:648] HybridClock initialized: now 1790506351614896 us; error 23 us; skew 500 ppm
Sep 27 10:52:31 dist-test-slave-p9q6 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506351, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:31.617538 28534 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:31.618376 28534 webserver.cc:460] Webserver started at http://127.9.39.190:34279/ using document root <none> and password file <none>
I20260927 10:52:31.618947 28534 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:31.619009 28534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:31.619164 28534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:31.621099 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "905efb5e8ef04286a84d31d808bd60df"
format_stamp: "Formatted at 2026-09-27 10:52:31 on dist-test-slave-p9q6"
server_key: "1fceac6f766b3ce32cceb9e6befc7194"
server_key_iv: "127ffb557068b533b01444b13783bdf5"
server_key_version: "encryptionkey@0"
I20260927 10:52:31.621590 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "905efb5e8ef04286a84d31d808bd60df"
format_stamp: "Formatted at 2026-09-27 10:52:31 on dist-test-slave-p9q6"
server_key: "1fceac6f766b3ce32cceb9e6befc7194"
server_key_iv: "127ffb557068b533b01444b13783bdf5"
server_key_version: "encryptionkey@0"
I20260927 10:52:31.625133 28534 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:52:31.627319 28549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:31.628486 28534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:31.628619 28534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "905efb5e8ef04286a84d31d808bd60df"
format_stamp: "Formatted at 2026-09-27 10:52:31 on dist-test-slave-p9q6"
server_key: "1fceac6f766b3ce32cceb9e6befc7194"
server_key_iv: "127ffb557068b533b01444b13783bdf5"
server_key_version: "encryptionkey@0"
I20260927 10:52:31.628724 28534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:31.638674 28534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:31.641933 28534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:31.642077 28534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:31.649240 28534 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:41745
I20260927 10:52:31.649256 28601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:41745 every 8 connection(s)
I20260927 10:52:31.650161 28534 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:31.652726 28602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:31.657850 28602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df: Bootstrap starting.
I20260927 10:52:31.660045 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28534
I20260927 10:52:31.660182 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:31.660390 28602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:31.660517 9374 external_mini_cluster.cc:1428] Setting key 35e486455c4116c906e493cc94d65bbe
I20260927 10:52:31.661201 28602 log.cc:824] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:31.663190 28602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df: No bootstrap required, opened a new log
I20260927 10:52:31.666617 28602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905efb5e8ef04286a84d31d808bd60df" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41745 } }
I20260927 10:52:31.666904 28602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:31.666960 28602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 905efb5e8ef04286a84d31d808bd60df, State: Initialized, Role: FOLLOWER
I20260927 10:52:31.667424 28602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905efb5e8ef04286a84d31d808bd60df" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41745 } }
I20260927 10:52:31.667551 28602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:31.667630 28602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:31.667749 28602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:31.668625 28602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905efb5e8ef04286a84d31d808bd60df" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41745 } }
Sep 27 10:52:31 dist-test-slave-p9q6 krb5kdc[28521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:31.668984 28602 leader_election.cc:302] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 905efb5e8ef04286a84d31d808bd60df; no voters:
I20260927 10:52:31.669340 28602 leader_election.cc:288] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:31.669425 28607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:31.669592 28607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [term 1 LEADER]: Becoming Leader. State: Replica: 905efb5e8ef04286a84d31d808bd60df, State: Running, Role: LEADER
I20260927 10:52:31.669902 28607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905efb5e8ef04286a84d31d808bd60df" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41745 } }
I20260927 10:52:31.670596 28602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:31.671053 28607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 905efb5e8ef04286a84d31d808bd60df. Latest consensus state: current_term: 1 leader_uuid: "905efb5e8ef04286a84d31d808bd60df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905efb5e8ef04286a84d31d808bd60df" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41745 } } }
I20260927 10:52:31.671165 28607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:31.671540 28611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:31.672387 28608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "905efb5e8ef04286a84d31d808bd60df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905efb5e8ef04286a84d31d808bd60df" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 41745 } } }
I20260927 10:52:31.672498 28608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:31.674815 28611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:31.676074 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:31.661651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.9.39.190:41745)
0927 10:52:31.662011 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:31.662029 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:31.662074 (+ 45us) server_negotiation.cc:407] Connection header received
0927 10:52:31.662762 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:31.662790 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:31.663104 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:31.663436 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:31.664253 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:31.665202 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:31.666056 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:31.666599 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:31.666838 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:31.669610 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:31.669618 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:31.669637 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:31.669672 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:31.671854 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:31.673044 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:31.673049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:31.673055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:31.673204 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:31.673578 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:31.673582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:31.673584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:31.674092 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:31.674302 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:31.674608 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 10:52:31.675332 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
I20260927 10:52:31.681900 28611 catalog_manager.cc:1347] Generated new cluster ID: d571c2c71dba4adba3fa9fe5378759a7
I20260927 10:52:31.681999 28611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:31.689601 28611 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:31.690521 28611 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:31.702448 28611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df: Generated new TSK 0
I20260927 10:52:31.703383 28611 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.39.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
Entry for principal kudu/127.9.39.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab.
I20260927 10:52:31.746510 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41745
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:31.871402 28628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:31.871646 28628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:31.871687 28628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:31.875702 28628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:31.875753 28628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:31.875809 28628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:31.880872 28628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.129.keytab
--principal=kudu/127.9.39.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:31.881769 28628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:31.882676 28628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:31.891985 28633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:31.891987 28634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:31.892028 28628 server_base.cc:1034] running on GCE node
W20260927 10:52:31.892047 28636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:31.892622 28628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:31.893169 28628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:31.894332 28628 hybrid_clock.cc:648] HybridClock initialized: now 1790506351894300 us; error 44 us; skew 500 ppm
Sep 27 10:52:31 dist-test-slave-p9q6 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506351, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:31.897015 28628 init.cc:377] Logged in from keytab as kudu/127.9.39.129@KRBTEST.COM (short username kudu)
I20260927 10:52:31.897851 28628 webserver.cc:460] Webserver started at http://127.9.39.129:40633/ using document root <none> and password file <none>
I20260927 10:52:31.898406 28628 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:31.898466 28628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:31.898772 28628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:31.900741 28628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "fd22ad15c73348e2b6d0f965fb2b9465"
format_stamp: "Formatted at 2026-09-27 10:52:31 on dist-test-slave-p9q6"
server_key: "c47f742d9e33d999f2e38145348110a9"
server_key_iv: "bc750bce8856e493b0cbf55801cfa8b7"
server_key_version: "encryptionkey@0"
I20260927 10:52:31.901283 28628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "fd22ad15c73348e2b6d0f965fb2b9465"
format_stamp: "Formatted at 2026-09-27 10:52:31 on dist-test-slave-p9q6"
server_key: "c47f742d9e33d999f2e38145348110a9"
server_key_iv: "bc750bce8856e493b0cbf55801cfa8b7"
server_key_version: "encryptionkey@0"
I20260927 10:52:31.904881 28628 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260927 10:52:31.907222 28643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:31.908388 28628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:31.908521 28628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "fd22ad15c73348e2b6d0f965fb2b9465"
format_stamp: "Formatted at 2026-09-27 10:52:31 on dist-test-slave-p9q6"
server_key: "c47f742d9e33d999f2e38145348110a9"
server_key_iv: "bc750bce8856e493b0cbf55801cfa8b7"
server_key_version: "encryptionkey@0"
I20260927 10:52:31.908648 28628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:31.921366 28628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:31.924325 28628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:31.924525 28628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:31.925089 28628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:31.925875 28628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:31.925945 28628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:31.926015 28628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:31.926059 28628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:31.935261 28628 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:46469
I20260927 10:52:31.935290 28756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:46469 every 8 connection(s)
I20260927 10:52:31.936111 28628 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:31.943775 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28628
I20260927 10:52:31.943926 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:52:31.944237 9374 external_mini_cluster.cc:1428] Setting key ee555e07b419f3b3d8c9ab6f1eab3a83
Sep 27 10:52:31 dist-test-slave-p9q6 krb5kdc[28521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506351, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.129@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:31.949761 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:31.937805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:44765 (local address 127.9.39.190:41745)
0927 10:52:31.937959 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:31.937964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:31.938765 (+ 801us) server_negotiation.cc:407] Connection header received
0927 10:52:31.940011 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:31.940014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:31.940060 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:31.940133 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:31.941268 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:31.941805 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:31.942972 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:31.943207 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:31.943288 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:31.945952 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:31.945956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:31.945958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:31.945991 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:31.947470 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:31.947988 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:31.947993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:31.947994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:31.948076 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:31.948429 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:31.948435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:31.948440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:31.948696 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:31.948800 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:31.949343 (+ 543us) server_negotiation.cc:299] Negotiation successful
0927 10:52:31.949493 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:52:31.950549 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:31.938172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:41745 (local address 127.9.39.129:44765)
0927 10:52:31.938602 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:31.938643 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:31.939821 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:31.940306 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:31.940312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:31.940662 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:31.941136 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:31.941149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:31.941920 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:31.941922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:31.942868 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:31.942877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:31.943334 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:31.943337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:31.943466 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:31.944116 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:31.944132 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:31.945791 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:31.947593 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:31.947599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:31.947614 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:31.947882 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:31.948172 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:31.948176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:31.948178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:31.948332 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:31.948849 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:31.948860 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:31.949096 (+ 236us) client_negotiation.cc:769] Sending connection context
0927 10:52:31.949262 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 10:52:31.949514 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
I20260927 10:52:31.951849 28757 heartbeater.cc:347] Connected to a master server at 127.9.39.190:41745
I20260927 10:52:31.952107 28757 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:31.952710 28757 heartbeater.cc:511] Master 127.9.39.190:41745 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.39.130@KRBTEST.COM; defaulting to no policy
I20260927 10:52:31.954274 28566 ts_manager.cc:194] Registered new tserver with Master: fd22ad15c73348e2b6d0f965fb2b9465 (127.9.39.129:46469)
I20260927 10:52:31.955509 28566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.129@KRBTEST.COM'} at 127.9.39.129:44765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
Entry for principal kudu/127.9.39.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab.
I20260927 10:52:31.973702 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41745
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:32.082732 28762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:32.083034 28762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:32.083110 28762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:32.087208 28762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:32.087291 28762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:32.087378 28762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:32.092830 28762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.130.keytab
--principal=kudu/127.9.39.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:32.093930 28762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:32.095029 28762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:32.104573 28770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.104573 28767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.104584 28768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:32.104954 28762 server_base.cc:1034] running on GCE node
I20260927 10:52:32.105319 28762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:32.105921 28762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:32.107074 28762 hybrid_clock.cc:648] HybridClock initialized: now 1790506352107054 us; error 28 us; skew 500 ppm
Sep 27 10:52:32 dist-test-slave-p9q6 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506352, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:32.109819 28762 init.cc:377] Logged in from keytab as kudu/127.9.39.130@KRBTEST.COM (short username kudu)
I20260927 10:52:32.110774 28762 webserver.cc:460] Webserver started at http://127.9.39.130:37573/ using document root <none> and password file <none>
I20260927 10:52:32.111354 28762 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:32.111413 28762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:32.111630 28762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:32.113642 28762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "115463cbe21244ad838d58ec59ae8cf1"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "2dcb6fc7ab0acbe834b1dc30aaf7e9e0"
server_key_iv: "94ae4ab7c5c15be1e9e64c9fe3c2bbd9"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.114181 28762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "115463cbe21244ad838d58ec59ae8cf1"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "2dcb6fc7ab0acbe834b1dc30aaf7e9e0"
server_key_iv: "94ae4ab7c5c15be1e9e64c9fe3c2bbd9"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.117715 28762 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:52:32.120218 28777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.121441 28762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:32.121563 28762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "115463cbe21244ad838d58ec59ae8cf1"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "2dcb6fc7ab0acbe834b1dc30aaf7e9e0"
server_key_iv: "94ae4ab7c5c15be1e9e64c9fe3c2bbd9"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.121660 28762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:32.156810 28762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:32.159920 28762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:32.160130 28762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:32.160759 28762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:32.161576 28762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:32.161621 28762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.161660 28762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:32.161718 28762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.170397 28762 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:41537
I20260927 10:52:32.170423 28890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:41537 every 8 connection(s)
I20260927 10:52:32.171402 28762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:52:32.179862 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28762
I20260927 10:52:32.180001 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:52:32.180279 9374 external_mini_cluster.cc:1428] Setting key 07e145ed8120e1c21e9bf61a80ddc3ca
Sep 27 10:52:32 dist-test-slave-p9q6 krb5kdc[28521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506352, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.130@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:32.185153 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.173313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:34887 (local address 127.9.39.190:41745)
0927 10:52:32.173446 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.173450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.174266 (+ 816us) server_negotiation.cc:407] Connection header received
0927 10:52:32.175743 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.175746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.175802 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.175882 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:32.177014 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.177535 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.178662 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.178917 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.179028 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.181667 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:32.181671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:32.181673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:32.181702 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:32.183169 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:32.183651 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:32.183656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:32.183657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:32.183736 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:32.184060 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:32.184064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:32.184069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:32.184266 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:32.184369 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.184870 (+ 501us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.184991 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:52:32.186026 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.173690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:41745 (local address 127.9.39.130:34887)
0927 10:52:32.174114 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.174160 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.175525 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.176049 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.176057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.176410 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:32.176866 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.176878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.177667 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.177675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.178538 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.178547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.179073 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.179077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.179239 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.179835 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:32.179850 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:32.181506 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:32.183280 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:32.183286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:32.183300 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:32.183549 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:32.183822 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:32.183827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:32.183829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:32.183968 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:32.184376 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:32.184384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:32.184634 (+ 250us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.184820 (+ 186us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.185143 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":95,"threads_started":1}
I20260927 10:52:32.187186 28891 heartbeater.cc:347] Connected to a master server at 127.9.39.190:41745
I20260927 10:52:32.187429 28891 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:32.187919 28891 heartbeater.cc:511] Master 127.9.39.190:41745 requested a full tablet report, sending...
I20260927 10:52:32.188982 28566 ts_manager.cc:194] Registered new tserver with Master: 115463cbe21244ad838d58ec59ae8cf1 (127.9.39.130:41537)
WARNING: no policy specified for kudu/127.9.39.131@KRBTEST.COM; defaulting to no policy
I20260927 10:52:32.189759 28566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.130@KRBTEST.COM'} at 127.9.39.130:34887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
Entry for principal kudu/127.9.39.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab.
I20260927 10:52:32.207116 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41745
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:32.312882 28896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:32.313148 28896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:32.313194 28896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:32.317159 28896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:32.317212 28896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:32.317268 28896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:32.322253 28896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.131.keytab
--principal=kudu/127.9.39.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:41745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:32.323271 28896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:32.324239 28896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:32.333825 28901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.333859 28902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.333832 28904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:32.334123 28896 server_base.cc:1034] running on GCE node
I20260927 10:52:32.334484 28896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:32.335170 28896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:32.336326 28896 hybrid_clock.cc:648] HybridClock initialized: now 1790506352336294 us; error 44 us; skew 500 ppm
Sep 27 10:52:32 dist-test-slave-p9q6 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506352, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:32.339032 28896 init.cc:377] Logged in from keytab as kudu/127.9.39.131@KRBTEST.COM (short username kudu)
I20260927 10:52:32.339850 28896 webserver.cc:460] Webserver started at http://127.9.39.131:41771/ using document root <none> and password file <none>
I20260927 10:52:32.340404 28896 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:32.340497 28896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:32.340740 28896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:32.342808 28896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "62d14e9589324dc2b41dfde02a52a86e"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "6187af3897216f6757b7ebab29fefd97"
server_key_iv: "a289c0f66093e3f02b84e19a11be5130"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.343354 28896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "62d14e9589324dc2b41dfde02a52a86e"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "6187af3897216f6757b7ebab29fefd97"
server_key_iv: "a289c0f66093e3f02b84e19a11be5130"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.347406 28896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260927 10:52:32.355984 28911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.357319 28896 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260927 10:52:32.357450 28896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "62d14e9589324dc2b41dfde02a52a86e"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "6187af3897216f6757b7ebab29fefd97"
server_key_iv: "a289c0f66093e3f02b84e19a11be5130"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.357580 28896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:32.384392 28896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:32.387717 28896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:32.387921 28896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:32.388512 28896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:32.389333 28896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:32.389402 28896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.389474 28896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:32.389528 28896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.398479 28896 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:44647
I20260927 10:52:32.398507 29024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:44647 every 8 connection(s)
I20260927 10:52:32.399410 28896 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:32.405669 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 28896
I20260927 10:52:32.405823 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:52:32.406122 9374 external_mini_cluster.cc:1428] Setting key 4bad8512bd0b454d7d9dc18103d4d7bd
Sep 27 10:52:32 dist-test-slave-p9q6 krb5kdc[28521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506352, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.131@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:32.413100 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.401277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:44979 (local address 127.9.39.190:41745)
0927 10:52:32.401412 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.401417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.402193 (+ 776us) server_negotiation.cc:407] Connection header received
0927 10:52:32.403629 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.403632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.403688 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.403767 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:32.404898 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.405438 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.406876 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.407183 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.407265 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.409869 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:32.409872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:32.409874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:32.409898 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:32.411349 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:32.411780 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:32.411783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:32.411784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:32.411859 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:32.412194 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:32.412197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:32.412198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:32.412368 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:32.412444 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.412890 (+ 446us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.412983 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:52:32.413894 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.401643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:41745 (local address 127.9.39.131:44979)
0927 10:52:32.402049 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.402089 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.403424 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.403929 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.403935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.404287 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:32.404752 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.404766 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.405563 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.405567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.406752 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.406762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.407338 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.407342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.407524 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.408151 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:52:32.408175 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:32.409714 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:32.411442 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:32.411446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:32.411457 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:32.411699 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:32.411937 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:52:32.411940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:52:32.411941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:52:32.412110 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:52:32.412454 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:32.412458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:32.412688 (+ 230us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.412839 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.413113 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":101,"threads_started":1}
I20260927 10:52:32.414973 29025 heartbeater.cc:347] Connected to a master server at 127.9.39.190:41745
I20260927 10:52:32.415210 29025 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:32.415664 29025 heartbeater.cc:511] Master 127.9.39.190:41745 requested a full tablet report, sending...
I20260927 10:52:32.416604 28566 ts_manager.cc:194] Registered new tserver with Master: 62d14e9589324dc2b41dfde02a52a86e (127.9.39.131:44647)
I20260927 10:52:32.417137 28566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.39.131@KRBTEST.COM'} at 127.9.39.131:44979
I20260927 10:52:32.420111 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:52:32.428802 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.421971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.9.39.190:41745)
0927 10:52:32.422107 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.422112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.422301 (+ 189us) server_negotiation.cc:407] Connection header received
0927 10:52:32.422408 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.422410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.422445 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.422508 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:32.423199 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.423668 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.424489 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.424707 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.424779 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.425998 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:32.426001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:32.426003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:32.426023 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:32.427404 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:32.427747 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:32.427749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:32.427751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:32.427817 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:32.428026 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:32.428028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:32.428029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:32.428221 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:32.428321 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.428499 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.428581 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:52:32.431357 28566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:52:32.434443 28566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:52:32.448163 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.443247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46469 (local address 127.0.0.1:41282)
0927 10:52:32.443746 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.443759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.443858 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.444289 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.444294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.444319 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:32.444457 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.444463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.445523 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.445530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.446797 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.446806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.447651 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.447654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.447767 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.447809 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.447889 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.447997 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260927 10:52:32.448633 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.443131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41537 (local address 127.0.0.1:35028)
0927 10:52:32.443416 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.443429 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.443514 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.444435 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.444441 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.444464 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:32.444598 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.444606 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.446070 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.446074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.447337 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.447348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.448147 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.448151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.448348 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.448370 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.448433 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.448506 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":64,"thread_start_us":96,"threads_started":1}
I20260927 10:52:32.448812 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.442393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:44647 (local address 127.0.0.1:34344)
0927 10:52:32.442798 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.442850 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.442985 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.444530 (+ 1545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.444535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.444556 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:32.444645 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.444652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.446081 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.446083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.447337 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.447348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.448421 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.448425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.448535 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.448559 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.448608 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.448674 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":29,"thread_start_us":72,"threads_started":1}
I20260927 10:52:32.448899 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.443335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35028 (local address 127.9.39.130:41537)
0927 10:52:32.443770 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.443785 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.443809 (+ 24us) server_negotiation.cc:407] Connection header received
0927 10:52:32.443887 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.443894 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.444104 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.444273 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:32.444760 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.445918 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.447498 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.448018 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.448141 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.448291 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.448552 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.448642 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":118,"threads_started":1}
I20260927 10:52:32.449397 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.442574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.9.39.131:44647)
0927 10:52:32.444119 (+ 1545us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.444130 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.444150 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:52:32.444235 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.444240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.444398 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.444536 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:32.444769 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.445923 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.447466 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.448284 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.448974 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.449121 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.449168 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.449260 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1429,"thread_start_us":181,"threads_started":1}
I20260927 10:52:32.449694 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.442945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.9.39.129:46469)
0927 10:52:32.443630 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.443645 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.443847 (+ 202us) server_negotiation.cc:407] Connection header received
0927 10:52:32.443966 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.443970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.444110 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.444275 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:32.444603 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.445399 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.446917 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.447516 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.448537 (+ 1021us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.448675 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.448721 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.448800 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":95,"threads_started":1}
I20260927 10:52:32.451876 28959 tablet_service.cc:1467] Processing CreateTablet for tablet 759ab66e12664379bc0ff37f633e49cb (DEFAULT_TABLE table=test-table [id=708383299c244f6fb15baf7de70a5a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:32.451879 28825 tablet_service.cc:1467] Processing CreateTablet for tablet 759ab66e12664379bc0ff37f633e49cb (DEFAULT_TABLE table=test-table [id=708383299c244f6fb15baf7de70a5a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:32.452174 28691 tablet_service.cc:1467] Processing CreateTablet for tablet 759ab66e12664379bc0ff37f633e49cb (DEFAULT_TABLE table=test-table [id=708383299c244f6fb15baf7de70a5a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:32.452790 28959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759ab66e12664379bc0ff37f633e49cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:32.452872 28825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759ab66e12664379bc0ff37f633e49cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:32.452958 28691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759ab66e12664379bc0ff37f633e49cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:32.458886 29042 tablet_bootstrap.cc:492] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Bootstrap starting.
I20260927 10:52:32.460202 29044 tablet_bootstrap.cc:492] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Bootstrap starting.
I20260927 10:52:32.460359 29043 tablet_bootstrap.cc:492] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: Bootstrap starting.
I20260927 10:52:32.461732 29042 tablet_bootstrap.cc:654] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:32.462267 29043 tablet_bootstrap.cc:654] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:32.462651 29044 tablet_bootstrap.cc:654] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:32.462949 29042 log.cc:824] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:32.463138 29043 log.cc:824] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:32.463405 29044 log.cc:824] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:32.465128 29042 tablet_bootstrap.cc:492] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: No bootstrap required, opened a new log
I20260927 10:52:32.465183 29043 tablet_bootstrap.cc:492] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: No bootstrap required, opened a new log
I20260927 10:52:32.465322 29043 ts_tablet_manager.cc:1397] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:52:32.465322 29042 ts_tablet_manager.cc:1397] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260927 10:52:32.465431 29044 tablet_bootstrap.cc:492] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: No bootstrap required, opened a new log
I20260927 10:52:32.465669 29044 ts_tablet_manager.cc:1397] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260927 10:52:32.469455 29044 raft_consensus.cc:348] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.469408 29042 raft_consensus.cc:348] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.469727 29042 raft_consensus.cc:374] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:32.469729 29044 raft_consensus.cc:374] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:32.469789 29044 raft_consensus.cc:729] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115463cbe21244ad838d58ec59ae8cf1, State: Initialized, Role: FOLLOWER
I20260927 10:52:32.469795 29042 raft_consensus.cc:729] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d14e9589324dc2b41dfde02a52a86e, State: Initialized, Role: FOLLOWER
I20260927 10:52:32.470252 29044 consensus_queue.cc:260] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.470256 29042 consensus_queue.cc:260] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.471211 28891 heartbeater.cc:503] Master 127.9.39.190:41745 was elected leader, sending a full tablet report...
I20260927 10:52:32.471417 29044 ts_tablet_manager.cc:1428] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 10:52:32.471418 29042 ts_tablet_manager.cc:1428] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 10:52:32.471798 29025 heartbeater.cc:503] Master 127.9.39.190:41745 was elected leader, sending a full tablet report...
I20260927 10:52:32.472184 29043 raft_consensus.cc:348] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.472438 29043 raft_consensus.cc:374] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:32.472499 29043 raft_consensus.cc:729] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd22ad15c73348e2b6d0f965fb2b9465, State: Initialized, Role: FOLLOWER
I20260927 10:52:32.472994 29043 consensus_queue.cc:260] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.473910 28757 heartbeater.cc:503] Master 127.9.39.190:41745 was elected leader, sending a full tablet report...
I20260927 10:52:32.473918 29043 ts_tablet_manager.cc:1428] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260927 10:52:32.480302 29050 raft_consensus.cc:482] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:32.480468 29050 raft_consensus.cc:504] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.481542 29050 leader_election.cc:288] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62d14e9589324dc2b41dfde02a52a86e (127.9.39.131:44647), 115463cbe21244ad838d58ec59ae8cf1 (127.9.39.130:41537)
I20260927 10:52:32.485481 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.481780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:44557 (local address 127.9.39.131:44647)
0927 10:52:32.481957 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.481963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.482082 (+ 119us) server_negotiation.cc:407] Connection header received
0927 10:52:32.482230 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.482233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.482287 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.482360 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:32.482789 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.483389 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.484519 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.484941 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.485038 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.485074 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.485285 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.485358 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260927 10:52:32.485536 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.481722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:44647 (local address 127.9.39.129:44557)
0927 10:52:32.481985 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.482000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.482104 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.482435 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.482437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.482452 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:32.482621 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.482628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.483525 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.483528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.484399 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.484408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.485052 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.485056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.485160 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.485185 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.485262 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.485335 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":109,"threads_started":1}
I20260927 10:52:32.485622 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.482272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:47219 (local address 127.9.39.130:41537)
0927 10:52:32.482382 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.482385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.482523 (+ 138us) server_negotiation.cc:407] Connection header received
0927 10:52:32.482614 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.482617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.482666 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.482779 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:32.483052 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.483582 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.484624 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.485074 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.485183 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.485237 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.485434 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.485496 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:52:32.485697 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.482116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41537 (local address 127.9.39.129:47219)
0927 10:52:32.482429 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.482443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.482544 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.482801 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.482804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.482821 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:32.482935 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.482941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.483689 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.483692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.484507 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.484516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.485185 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.485187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.485306 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.485328 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 10:52:32.485401 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 10:52:32.485463 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":92,"threads_started":1}
I20260927 10:52:32.486177 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759ab66e12664379bc0ff37f633e49cb" candidate_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d14e9589324dc2b41dfde02a52a86e" is_pre_election: true
I20260927 10:52:32.486177 28845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759ab66e12664379bc0ff37f633e49cb" candidate_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115463cbe21244ad838d58ec59ae8cf1" is_pre_election: true
I20260927 10:52:32.486404 28979 raft_consensus.cc:2454] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd22ad15c73348e2b6d0f965fb2b9465 in term 0.
I20260927 10:52:32.486423 28845 raft_consensus.cc:2454] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd22ad15c73348e2b6d0f965fb2b9465 in term 0.
I20260927 10:52:32.486871 28646 leader_election.cc:302] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d14e9589324dc2b41dfde02a52a86e, fd22ad15c73348e2b6d0f965fb2b9465; no voters:
I20260927 10:52:32.487134 29050 raft_consensus.cc:2781] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:52:32.487205 29050 raft_consensus.cc:482] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:52:32.487236 29050 raft_consensus.cc:3037] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:32.488236 29050 raft_consensus.cc:504] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.488564 29050 leader_election.cc:288] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [CANDIDATE]: Term 1 election: Requested vote from peers 62d14e9589324dc2b41dfde02a52a86e (127.9.39.131:44647), 115463cbe21244ad838d58ec59ae8cf1 (127.9.39.130:41537)
I20260927 10:52:32.488940 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759ab66e12664379bc0ff37f633e49cb" candidate_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d14e9589324dc2b41dfde02a52a86e"
I20260927 10:52:32.489029 28845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759ab66e12664379bc0ff37f633e49cb" candidate_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115463cbe21244ad838d58ec59ae8cf1"
I20260927 10:52:32.489104 28979 raft_consensus.cc:3037] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:32.489177 28845 raft_consensus.cc:3037] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:32.490181 28845 raft_consensus.cc:2454] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd22ad15c73348e2b6d0f965fb2b9465 in term 1.
I20260927 10:52:32.490182 28979 raft_consensus.cc:2454] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd22ad15c73348e2b6d0f965fb2b9465 in term 1.
I20260927 10:52:32.490463 28646 leader_election.cc:302] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d14e9589324dc2b41dfde02a52a86e, fd22ad15c73348e2b6d0f965fb2b9465; no voters:
I20260927 10:52:32.490650 29050 raft_consensus.cc:2781] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:32.490897 29050 raft_consensus.cc:686] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 1 LEADER]: Becoming Leader. State: Replica: fd22ad15c73348e2b6d0f965fb2b9465, State: Running, Role: LEADER
I20260927 10:52:32.491235 29050 consensus_queue.cc:237] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } }
I20260927 10:52:32.494732 28565 catalog_manager.cc:5495] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd22ad15c73348e2b6d0f965fb2b9465 (127.9.39.129). New cstate: current_term: 1 leader_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd22ad15c73348e2b6d0f965fb2b9465" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 46469 } health_report { overall_health: HEALTHY } } }
I20260927 10:52:32.513100 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.509640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41286 (local address 127.9.39.129:46469)
0927 10:52:32.509770 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.509774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.509786 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:52:32.509856 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.509858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.509910 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.509997 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:52:32.510220 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.510769 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.511628 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.511874 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.511951 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.512244 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:52:32.512676 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:52:32.512765 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.512905 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.512974 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:52:32.518777 28979 raft_consensus.cc:1261] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 1 FOLLOWER]: Refusing update from remote peer fd22ad15c73348e2b6d0f965fb2b9465: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:32.518808 28845 raft_consensus.cc:1261] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 1 FOLLOWER]: Refusing update from remote peer fd22ad15c73348e2b6d0f965fb2b9465: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:32.519510 29053 consensus_queue.cc:1035] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115463cbe21244ad838d58ec59ae8cf1" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 41537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:32.519701 29050 consensus_queue.cc:1035] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d14e9589324dc2b41dfde02a52a86e" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:32.527722 29058 mvcc.cc:205] Tried to move back new op lower bound from 7333914019909820416 to 7333914019807244288. Current Snapshot: MvccSnapshot[applied={T|T < 7333914019909820416}]
I20260927 10:52:32.527722 29057 mvcc.cc:205] Tried to move back new op lower bound from 7333914019909820416 to 7333914019807244288. Current Snapshot: MvccSnapshot[applied={T|T < 7333914019909820416}]
I20260927 10:52:32.537120 28565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:32.537324 28565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:32.539832 28565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 905efb5e8ef04286a84d31d808bd60df: Sending DeleteTablet for 3 replicas of tablet 759ab66e12664379bc0ff37f633e49cb
I20260927 10:52:32.540690 28959 tablet_service.cc:1514] Processing DeleteTablet for tablet 759ab66e12664379bc0ff37f633e49cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:32 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:34344
I20260927 10:52:32.540776 28691 tablet_service.cc:1514] Processing DeleteTablet for tablet 759ab66e12664379bc0ff37f633e49cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:32 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:41282
I20260927 10:52:32.541220 29066 tablet_replica.cc:330] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: stopping tablet replica
I20260927 10:52:32.541352 28825 tablet_service.cc:1514] Processing DeleteTablet for tablet 759ab66e12664379bc0ff37f633e49cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:32 UTC) from {username='kudu', principal='kudu/127.9.39.190@KRBTEST.COM'} at 127.0.0.1:35028
I20260927 10:52:32.541553 29066 raft_consensus.cc:2229] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:32.541862 29066 raft_consensus.cc:2258] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:32.542025 29067 tablet_replica.cc:330] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465: stopping tablet replica
I20260927 10:52:32.542271 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28628
I20260927 10:52:32.542323 29067 raft_consensus.cc:2229] T 759ab66e12664379bc0ff37f633e49cb P fd22ad15c73348e2b6d0f965fb2b9465 [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:52:32.543677 29068 tablet_replica.cc:330] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: stopping tablet replica
I20260927 10:52:32.543820 29066 ts_tablet_manager.cc:1905] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:32.543963 29068 raft_consensus.cc:2229] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:32.544278 29068 raft_consensus.cc:2258] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:32.545826 29068 ts_tablet_manager.cc:1905] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:32.546530 29066 ts_tablet_manager.cc:1918] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:32.546640 29066 log.cc:1196] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/759ab66e12664379bc0ff37f633e49cb
I20260927 10:52:32.546986 29066 ts_tablet_manager.cc:1939] T 759ab66e12664379bc0ff37f633e49cb P 62d14e9589324dc2b41dfde02a52a86e: Deleting consensus metadata
I20260927 10:52:32.548291 28552 catalog_manager.cc:4841] TS 62d14e9589324dc2b41dfde02a52a86e (127.9.39.131:44647): tablet 759ab66e12664379bc0ff37f633e49cb (table test-table [id=708383299c244f6fb15baf7de70a5a6c]) successfully deleted
I20260927 10:52:32.548530 29068 ts_tablet_manager.cc:1918] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:32.548619 29068 log.cc:1196] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/759ab66e12664379bc0ff37f633e49cb
I20260927 10:52:32.548977 29068 ts_tablet_manager.cc:1939] T 759ab66e12664379bc0ff37f633e49cb P 115463cbe21244ad838d58ec59ae8cf1: Deleting consensus metadata
W20260927 10:52:32.549280 28551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:46469 (error 108)
I20260927 10:52:32.549521 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28762
I20260927 10:52:32.551812 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.551453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:46469 (local address 127.0.0.1:41288)
0927 10:52:32.551611 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.551704 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:46469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 10:52:32.551995 28551 catalog_manager.cc:4584] TS fd22ad15c73348e2b6d0f965fb2b9465 (127.9.39.129:46469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 759ab66e12664379bc0ff37f633e49cb: Network error: Client connection negotiation failed: client connection to 127.9.39.129:46469: connect: Connection refused (error 111)
I20260927 10:52:32.555411 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28896
I20260927 10:52:32.555711 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.555442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:41537 (local address 127.0.0.1:35038)
0927 10:52:32.555565 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.555620 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.130:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 10:52:32.560987 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 28534
2026-09-27T10:52:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2233 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T10:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:32Z Disabled control of system clock
I20260927 10:52:32.591223 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:35015
--webserver_interface=127.9.39.190
--webserver_port=0
--builtin_ntp_servers=127.9.39.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:35015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 10:52:32.699193 29073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:32.699467 29073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:32.699504 29073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:32.703521 29073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:32.703593 29073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:32.703608 29073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:32.703795 29073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:32.703819 29073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:32.709048 29073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:35015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:35015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:32.710110 29073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:32.711458 29073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:32.719947 29079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:32.720053 29073 server_base.cc:1034] running on GCE node
W20260927 10:52:32.719918 29081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.719986 29078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:32.720927 29073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:32.722011 29073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:32.723165 29073 hybrid_clock.cc:648] HybridClock initialized: now 1790506352723147 us; error 31 us; skew 500 ppm
I20260927 10:52:32.724841 29073 webserver.cc:460] Webserver started at http://127.9.39.190:44849/ using document root <none> and password file <none>
I20260927 10:52:32.725361 29073 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:32.725417 29073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:32.725572 29073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:32.727586 29073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "c6554093d66145db95bcf9511f2f862f"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "0aea5cf5bd371b501fb8aa7fc223e053"
server_key_iv: "0b061b338002e65dfd0d211b83108b79"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.728083 29073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "c6554093d66145db95bcf9511f2f862f"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "0aea5cf5bd371b501fb8aa7fc223e053"
server_key_iv: "0b061b338002e65dfd0d211b83108b79"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.731549 29073 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:52:32.733781 29087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.735023 29073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:32.735157 29073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "c6554093d66145db95bcf9511f2f862f"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "0aea5cf5bd371b501fb8aa7fc223e053"
server_key_iv: "0b061b338002e65dfd0d211b83108b79"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.735257 29073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:32.750859 29073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:32.751447 29073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:32.751629 29073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:32.758383 29073 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:35015
I20260927 10:52:32.758390 29139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:35015 every 8 connection(s)
I20260927 10:52:32.759341 29073 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:32.762320 29140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:32.766883 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29073
I20260927 10:52:32.767030 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:32.767338 9374 external_mini_cluster.cc:1428] Setting key 20c076df971d317a35928055e809ca79
I20260927 10:52:32.767817 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f: Bootstrap starting.
I20260927 10:52:32.770366 29140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:32.771173 29140 log.cc:824] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:32.772935 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f: No bootstrap required, opened a new log
I20260927 10:52:32.776216 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.768738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.9.39.190:35015)
0927 10:52:32.769339 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.769352 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.769385 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:52:32.770006 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.770021 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.770402 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.770801 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:32.771211 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.772152 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.773033 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.773492 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.773661 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.774522 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:32.774526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:32.774538 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:32.774953 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:32.775004 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:32.775024 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:32.775153 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:32.775300 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 10:52:32.775541 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260927 10:52:32.776734 29140 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6554093d66145db95bcf9511f2f862f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35015 } }
I20260927 10:52:32.776966 29140 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:32.777041 29140 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6554093d66145db95bcf9511f2f862f, State: Initialized, Role: FOLLOWER
I20260927 10:52:32.777499 29140 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6554093d66145db95bcf9511f2f862f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35015 } }
I20260927 10:52:32.777658 29140 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:32.777738 29140 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:32.777863 29140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:32.778862 29140 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6554093d66145db95bcf9511f2f862f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35015 } }
I20260927 10:52:32.779202 29140 leader_election.cc:302] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6554093d66145db95bcf9511f2f862f; no voters:
I20260927 10:52:32.779507 29140 leader_election.cc:288] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:32.779671 29145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:32.779889 29145 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [term 1 LEADER]: Becoming Leader. State: Replica: c6554093d66145db95bcf9511f2f862f, State: Running, Role: LEADER
I20260927 10:52:32.780205 29145 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6554093d66145db95bcf9511f2f862f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35015 } }
I20260927 10:52:32.780812 29140 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:32.781983 29146 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6554093d66145db95bcf9511f2f862f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6554093d66145db95bcf9511f2f862f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35015 } } }
I20260927 10:52:32.781983 29147 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6554093d66145db95bcf9511f2f862f. Latest consensus state: current_term: 1 leader_uuid: "c6554093d66145db95bcf9511f2f862f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6554093d66145db95bcf9511f2f862f" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 35015 } } }
I20260927 10:52:32.782143 29146 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:32.782143 29147 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:32.782562 29154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:32.785244 29154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:32.790279 29154 catalog_manager.cc:1347] Generated new cluster ID: 4b537d82cf5c424ea03b489717e38ff6
I20260927 10:52:32.790342 29154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:32.800302 29154 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:32.801177 29154 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:32.808065 29154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f: Generated new TSK 0
I20260927 10:52:32.808681 29154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:32.817292 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.129:0
--local_ip_for_outbound_sockets=127.9.39.129
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35015
--builtin_ntp_servers=127.9.39.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 10:52:32.928593 29164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:32.928845 29164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:32.928882 29164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:32.932713 29164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:32.932755 29164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:32.932806 29164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.129
I20260927 10:52:32.937359 29164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.39.129
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.129
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:32.938449 29164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:32.939822 29164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:32.949090 29172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.949090 29170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:32.949118 29169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:32.949170 29164 server_base.cc:1034] running on GCE node
I20260927 10:52:32.949755 29164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:32.950363 29164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:32.951538 29164 hybrid_clock.cc:648] HybridClock initialized: now 1790506352951529 us; error 26 us; skew 500 ppm
I20260927 10:52:32.953305 29164 webserver.cc:460] Webserver started at http://127.9.39.129:39809/ using document root <none> and password file <none>
I20260927 10:52:32.953850 29164 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:32.953938 29164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:32.954160 29164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:32.956501 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data/instance:
uuid: "5dcd958be02a40e1bb5fddc245da8ab9"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "a6248a9704ed27924b40ffee06681644"
server_key_iv: "c9ccb66f510cf84c21d26fcdeba1d003"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.957080 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance:
uuid: "5dcd958be02a40e1bb5fddc245da8ab9"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "a6248a9704ed27924b40ffee06681644"
server_key_iv: "c9ccb66f510cf84c21d26fcdeba1d003"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.960626 29164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:52:32.962944 29178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.964068 29164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 10:52:32.964210 29164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal
uuid: "5dcd958be02a40e1bb5fddc245da8ab9"
format_stamp: "Formatted at 2026-09-27 10:52:32 on dist-test-slave-p9q6"
server_key: "a6248a9704ed27924b40ffee06681644"
server_key_iv: "c9ccb66f510cf84c21d26fcdeba1d003"
server_key_version: "encryptionkey@0"
I20260927 10:52:32.964325 29164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:32.978317 29164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:32.979044 29164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:32.979240 29164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:32.979786 29164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:32.980705 29164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:32.980778 29164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.980849 29164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:32.980899 29164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:32.989669 29164 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.129:34601
I20260927 10:52:32.989701 29291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.129:34601 every 8 connection(s)
I20260927 10:52:32.990548 29164 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/data/info.pb
I20260927 10:52:32.993366 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29164
I20260927 10:52:32.993494 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-0/wal/instance
I20260927 10:52:32.993830 9374 external_mini_cluster.cc:1428] Setting key 8c0ea0bd2ec70db8616ad5c42c423c6e
I20260927 10:52:32.995884 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.130:0
--local_ip_for_outbound_sockets=127.9.39.130
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35015
--builtin_ntp_servers=127.9.39.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 10:52:33.000486 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.992347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.129:33567 (local address 127.9.39.190:35015)
0927 10:52:32.992479 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:32.992482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:32.993616 (+ 1134us) server_negotiation.cc:407] Connection header received
0927 10:52:32.995424 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:32.995426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:32.995466 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:32.995534 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:32.996682 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.997202 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.998424 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.998650 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.998807 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.000023 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:33.000026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:33.000029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:33.000041 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:33.000061 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:33.000069 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:33.000123 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.000326 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.000377 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 10:52:33.001761 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:32.992735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:35015 (local address 127.9.39.129:33567)
0927 10:52:32.993467 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:32.993526 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:32.995190 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:32.995722 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:32.995735 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:32.995845 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:32.996542 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.996562 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.997329 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.997338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.998290 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:32.998303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:32.998840 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:32.998844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:32.999025 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:32.999686 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:32.999690 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:32.999748 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:32.999842 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:33.000163 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:33.000176 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:33.000183 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.000327 (+ 144us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.000665 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":100,"threads_started":1}
I20260927 10:52:33.003118 29292 heartbeater.cc:347] Connected to a master server at 127.9.39.190:35015
I20260927 10:52:33.003362 29292 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:33.003878 29292 heartbeater.cc:511] Master 127.9.39.190:35015 requested a full tablet report, sending...
I20260927 10:52:33.005373 29104 ts_manager.cc:194] Registered new tserver with Master: 5dcd958be02a40e1bb5fddc245da8ab9 (127.9.39.129:34601)
I20260927 10:52:33.006502 29104 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.129:33567
W20260927 10:52:33.105291 29295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:33.105573 29295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:33.105643 29295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:33.110185 29295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:33.110256 29295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:33.110344 29295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.130
I20260927 10:52:33.115145 29295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.39.130
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.130
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:33.116148 29295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:33.117479 29295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:33.126976 29300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:33.126955 29301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:33.126988 29295 server_base.cc:1034] running on GCE node
W20260927 10:52:33.126955 29303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:33.127491 29295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:33.128072 29295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:33.129251 29295 hybrid_clock.cc:648] HybridClock initialized: now 1790506353129242 us; error 29 us; skew 500 ppm
I20260927 10:52:33.130964 29295 webserver.cc:460] Webserver started at http://127.9.39.130:33901/ using document root <none> and password file <none>
I20260927 10:52:33.131533 29295 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:33.131623 29295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:33.131852 29295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:33.133793 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data/instance:
uuid: "568075cd551c4561a62489f12e17d61c"
format_stamp: "Formatted at 2026-09-27 10:52:33 on dist-test-slave-p9q6"
server_key: "c5dc77e12aadfb9758fbfb2581157aa0"
server_key_iv: "0d4464d8dda8de53dafb52fbdc0606d6"
server_key_version: "encryptionkey@0"
I20260927 10:52:33.134357 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance:
uuid: "568075cd551c4561a62489f12e17d61c"
format_stamp: "Formatted at 2026-09-27 10:52:33 on dist-test-slave-p9q6"
server_key: "c5dc77e12aadfb9758fbfb2581157aa0"
server_key_iv: "0d4464d8dda8de53dafb52fbdc0606d6"
server_key_version: "encryptionkey@0"
I20260927 10:52:33.137758 29295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:52:33.140115 29309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:33.141230 29295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:33.141356 29295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal
uuid: "568075cd551c4561a62489f12e17d61c"
format_stamp: "Formatted at 2026-09-27 10:52:33 on dist-test-slave-p9q6"
server_key: "c5dc77e12aadfb9758fbfb2581157aa0"
server_key_iv: "0d4464d8dda8de53dafb52fbdc0606d6"
server_key_version: "encryptionkey@0"
I20260927 10:52:33.141463 29295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:33.167188 29295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:33.167876 29295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:33.168128 29295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:33.168686 29295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:33.169668 29295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:33.169739 29295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:33.169811 29295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:33.169853 29295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:33.178651 29295 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.130:43939
I20260927 10:52:33.178684 29422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.130:43939 every 8 connection(s)
I20260927 10:52:33.179538 29295 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/data/info.pb
I20260927 10:52:33.182868 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29295
I20260927 10:52:33.182987 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal/instance
I20260927 10:52:33.183274 9374 external_mini_cluster.cc:1428] Setting key eff65dcb0087d1bd72d1d10fab3f508a
I20260927 10:52:33.185268 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.39.131:0
--local_ip_for_outbound_sockets=127.9.39.131
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35015
--builtin_ntp_servers=127.9.39.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 10:52:33.189586 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.181331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:57095 (local address 127.9.39.190:35015)
0927 10:52:33.181512 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.181517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.182661 (+ 1144us) server_negotiation.cc:407] Connection header received
0927 10:52:33.184338 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.184341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.184381 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.184454 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:33.185431 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.186279 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.187455 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.187851 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.187974 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.189009 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:33.189016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:33.189020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:33.189038 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:33.189064 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:33.189075 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:33.189163 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.189300 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.189377 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 10:52:33.190308 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.181821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:35015 (local address 127.9.39.130:57095)
0927 10:52:33.182476 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.182514 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.184113 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.184661 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.184670 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.184775 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:33.185263 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.185276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.186402 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.186404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.187332 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.187342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.187994 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.187997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.188137 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.188752 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:33.188753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:33.188791 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:33.188852 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:33.189161 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:33.189168 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:33.189173 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.189266 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.189478 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":115,"threads_started":1}
I20260927 10:52:33.191474 29423 heartbeater.cc:347] Connected to a master server at 127.9.39.190:35015
I20260927 10:52:33.191694 29423 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:33.192185 29423 heartbeater.cc:511] Master 127.9.39.190:35015 requested a full tablet report, sending...
I20260927 10:52:33.193208 29104 ts_manager.cc:194] Registered new tserver with Master: 568075cd551c4561a62489f12e17d61c (127.9.39.130:43939)
I20260927 10:52:33.193769 29104 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.130:57095
W20260927 10:52:33.296541 29426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:33.296795 29426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:33.296830 29426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:33.300881 29426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:33.300928 29426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:33.300998 29426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.39.131
I20260927 10:52:33.305747 29426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.39.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.39.131
--webserver_port=0
--tserver_master_addrs=127.9.39.190:35015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.39.131
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:33.306833 29426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:33.308357 29426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:33.318091 29431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:33.318145 29434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:33.318094 29432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:33.318434 29426 server_base.cc:1034] running on GCE node
I20260927 10:52:33.318929 29426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:33.319504 29426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:33.320686 29426 hybrid_clock.cc:648] HybridClock initialized: now 1790506353320623 us; error 42 us; skew 500 ppm
I20260927 10:52:33.322522 29426 webserver.cc:460] Webserver started at http://127.9.39.131:42791/ using document root <none> and password file <none>
I20260927 10:52:33.323258 29426 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:33.323343 29426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:33.323539 29426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:33.325476 29426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data/instance:
uuid: "ad7731e6de7b4d8d9f6625922342af1f"
format_stamp: "Formatted at 2026-09-27 10:52:33 on dist-test-slave-p9q6"
server_key: "bf7b9c6b4c26363dc06d3bbd4504ed78"
server_key_iv: "ddd93d992a09e8c6480f888963a62e9f"
server_key_version: "encryptionkey@0"
I20260927 10:52:33.325986 29426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance:
uuid: "ad7731e6de7b4d8d9f6625922342af1f"
format_stamp: "Formatted at 2026-09-27 10:52:33 on dist-test-slave-p9q6"
server_key: "bf7b9c6b4c26363dc06d3bbd4504ed78"
server_key_iv: "ddd93d992a09e8c6480f888963a62e9f"
server_key_version: "encryptionkey@0"
I20260927 10:52:33.329589 29426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 10:52:33.332031 29440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:33.333237 29426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:33.333326 29426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal
uuid: "ad7731e6de7b4d8d9f6625922342af1f"
format_stamp: "Formatted at 2026-09-27 10:52:33 on dist-test-slave-p9q6"
server_key: "bf7b9c6b4c26363dc06d3bbd4504ed78"
server_key_iv: "ddd93d992a09e8c6480f888963a62e9f"
server_key_version: "encryptionkey@0"
I20260927 10:52:33.333457 29426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:33.370575 29426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:33.371311 29426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:33.371512 29426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:33.372104 29426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 10:52:33.373049 29426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 10:52:33.373095 29426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:33.373162 29426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 10:52:33.373205 29426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:33.382414 29426 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.131:43329
I20260927 10:52:33.382422 29553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.131:43329 every 8 connection(s)
I20260927 10:52:33.383459 29426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/data/info.pb
I20260927 10:52:33.391912 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.385253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.131:46923 (local address 127.9.39.190:35015)
0927 10:52:33.385383 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.385386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.386348 (+ 962us) server_negotiation.cc:407] Connection header received
0927 10:52:33.387647 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.387650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.387696 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.387767 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:33.388616 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.389117 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.390185 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.390429 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.390521 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.391514 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:33.391517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:33.391519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:33.391531 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:33.391549 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:33.391557 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:33.391614 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.391752 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.391800 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:52:33.392232 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29426
I20260927 10:52:33.392349 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal/instance
I20260927 10:52:33.392637 9374 external_mini_cluster.cc:1428] Setting key 9551b641660c1c17ea4711976f2ec752
I20260927 10:52:33.392942 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.385598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:35015 (local address 127.9.39.131:46923)
0927 10:52:33.386231 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.386268 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.387470 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.387948 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.387955 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.388033 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:52:33.388499 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.388510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.389213 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.389215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.390077 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.390085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.390567 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.390570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.390730 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.391280 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 10:52:33.391281 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:52:33.391318 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 10:52:33.391378 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:52:33.391636 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:52:33.391641 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:52:33.391645 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.391736 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.391938 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
I20260927 10:52:33.394039 29554 heartbeater.cc:347] Connected to a master server at 127.9.39.190:35015
I20260927 10:52:33.394279 29554 heartbeater.cc:464] Registering TS with master...
I20260927 10:52:33.394871 29554 heartbeater.cc:511] Master 127.9.39.190:35015 requested a full tablet report, sending...
I20260927 10:52:33.395833 29104 ts_manager.cc:194] Registered new tserver with Master: ad7731e6de7b4d8d9f6625922342af1f (127.9.39.131:43329)
I20260927 10:52:33.396395 29104 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.39.131:46923
I20260927 10:52:33.406301 9374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 10:52:33.411584 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.408252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.9.39.190:35015)
0927 10:52:33.408383 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.408387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.408546 (+ 159us) server_negotiation.cc:407] Connection header received
0927 10:52:33.408646 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.408648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.408688 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.408754 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:33.408989 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.409489 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.410270 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.410470 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.410542 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.411276 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:33.411279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:33.411282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:33.411292 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:33.411307 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:33.411315 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:33.411372 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.411439 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.411483 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:52:33.414276 29104 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 10:52:33.417198 29104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 10:52:33.431169 29570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.426119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.9.39.129:34601)
0927 10:52:33.426566 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.426576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.426594 (+ 18us) server_negotiation.cc:407] Connection header received
0927 10:52:33.426655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.426660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.426859 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.426979 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:33.427393 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.428480 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.429578 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.430411 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.430521 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.430722 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.430904 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.430999 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":94,"threads_started":1}
I20260927 10:52:33.431377 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.425732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:34601 (local address 127.0.0.1:36248)
0927 10:52:33.426329 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.426343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.426425 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.427157 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.427160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.427178 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:33.427270 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.427276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.428595 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.428598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.429455 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.429464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.430533 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.430537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.430682 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.430748 (+ 66us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.430862 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.431026 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":82,"spinlock_wait_cycles":5248,"thread_start_us":81,"threads_started":1}
I20260927 10:52:33.431856 29569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.425709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.9.39.130:43939)
0927 10:52:33.426446 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.426459 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.426482 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:52:33.426559 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.426565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.426775 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.426928 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:33.427110 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.427922 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.429696 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.430411 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.430521 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.430721 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.431602 (+ 881us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.431702 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":118,"threads_started":1}
I20260927 10:52:33.432284 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.425032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.130:43939 (local address 127.0.0.1:45840)
0927 10:52:33.426087 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.426101 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.426203 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.426910 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.426914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.426934 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:33.427014 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.427020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.428038 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.428042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.429577 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.429588 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.430533 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.430537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.430721 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.430748 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.430862 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.431026 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":128,"thread_start_us":88,"threads_started":1}
I20260927 10:52:33.432641 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.424954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43329 (local address 127.0.0.1:40152)
0927 10:52:33.425300 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.425352 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.425474 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.426372 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.426382 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.426413 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:33.426558 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.426563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.428102 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.428105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.429577 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.429587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.432246 (+ 2659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.432250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.432346 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.432373 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.432428 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.432500 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":95,"spinlock_wait_cycles":384,"thread_start_us":69,"threads_started":1}
I20260927 10:52:33.433306 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.425081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.9.39.131:43329)
0927 10:52:33.425829 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.425842 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.425867 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:52:33.425941 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.425946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.426185 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.426680 (+ 495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:33.426877 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.427966 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.430787 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.432106 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.432885 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.433061 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.433112 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.433191 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":133,"threads_started":1}
I20260927 10:52:33.435205 29357 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2e059ed1434e918368dc98bc4ae535 (DEFAULT_TABLE table=test-table [id=671c7001e37341f2be5151d1cd96ac3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:33.435205 29226 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2e059ed1434e918368dc98bc4ae535 (DEFAULT_TABLE table=test-table [id=671c7001e37341f2be5151d1cd96ac3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:33.435475 29488 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2e059ed1434e918368dc98bc4ae535 (DEFAULT_TABLE table=test-table [id=671c7001e37341f2be5151d1cd96ac3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 10:52:33.436357 29226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f2e059ed1434e918368dc98bc4ae535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:33.436357 29357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f2e059ed1434e918368dc98bc4ae535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:33.436509 29488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f2e059ed1434e918368dc98bc4ae535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:33.442787 29571 tablet_bootstrap.cc:492] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Bootstrap starting.
I20260927 10:52:33.444268 29572 tablet_bootstrap.cc:492] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Bootstrap starting.
I20260927 10:52:33.444892 29571 tablet_bootstrap.cc:654] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:33.445653 29571 log.cc:824] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:33.446148 29572 tablet_bootstrap.cc:654] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:33.446971 29572 log.cc:824] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:33.447042 29573 tablet_bootstrap.cc:492] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: Bootstrap starting.
I20260927 10:52:33.447810 29571 tablet_bootstrap.cc:492] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: No bootstrap required, opened a new log
I20260927 10:52:33.448005 29571 ts_tablet_manager.cc:1397] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 10:52:33.448910 29572 tablet_bootstrap.cc:492] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: No bootstrap required, opened a new log
I20260927 10:52:33.449021 29573 tablet_bootstrap.cc:654] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:33.449090 29572 ts_tablet_manager.cc:1397] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 10:52:33.449819 29573 log.cc:824] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:33.451370 29571 raft_consensus.cc:348] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.451628 29571 raft_consensus.cc:374] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:33.451683 29571 raft_consensus.cc:729] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7731e6de7b4d8d9f6625922342af1f, State: Initialized, Role: FOLLOWER
I20260927 10:52:33.452144 29571 consensus_queue.cc:260] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.452277 29573 tablet_bootstrap.cc:492] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: No bootstrap required, opened a new log
I20260927 10:52:33.452440 29573 ts_tablet_manager.cc:1397] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260927 10:52:33.452828 29554 heartbeater.cc:503] Master 127.9.39.190:35015 was elected leader, sending a full tablet report...
I20260927 10:52:33.452708 29572 raft_consensus.cc:348] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.452932 29572 raft_consensus.cc:374] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:33.452981 29572 raft_consensus.cc:729] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 568075cd551c4561a62489f12e17d61c, State: Initialized, Role: FOLLOWER
I20260927 10:52:33.453354 29572 consensus_queue.cc:260] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.453557 29571 ts_tablet_manager.cc:1428] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:52:33.454255 29572 ts_tablet_manager.cc:1428] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 10:52:33.454556 29423 heartbeater.cc:503] Master 127.9.39.190:35015 was elected leader, sending a full tablet report...
I20260927 10:52:33.456475 29573 raft_consensus.cc:348] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.456725 29573 raft_consensus.cc:374] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:33.456785 29573 raft_consensus.cc:729] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dcd958be02a40e1bb5fddc245da8ab9, State: Initialized, Role: FOLLOWER
I20260927 10:52:33.457218 29573 consensus_queue.cc:260] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.457847 29292 heartbeater.cc:503] Master 127.9.39.190:35015 was elected leader, sending a full tablet report...
I20260927 10:52:33.458077 29573 ts_tablet_manager.cc:1428] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260927 10:52:33.491737 29293 tablet.cc:2369] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:52:33.536226 29578 raft_consensus.cc:482] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 10:52:33.536441 29578 raft_consensus.cc:504] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.537722 29578 leader_election.cc:288] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad7731e6de7b4d8d9f6625922342af1f (127.9.39.131:43329), 5dcd958be02a40e1bb5fddc245da8ab9 (127.9.39.129:34601)
I20260927 10:52:33.541682 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.537919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.131:43329 (local address 127.9.39.130:54273)
0927 10:52:33.538043 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.538058 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.538155 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.538749 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.538752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.538771 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:33.538895 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.538901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.539718 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.539723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.540600 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.540609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.541303 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.541307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.541396 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.541412 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.541506 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.541569 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":6}
I20260927 10:52:33.541689 29570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.538107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:33313 (local address 127.9.39.129:34601)
0927 10:52:33.538263 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.538268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.538529 (+ 261us) server_negotiation.cc:407] Connection header received
0927 10:52:33.538678 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.538682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.538795 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.538874 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:33.539148 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.539699 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.540729 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.541110 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.541204 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.541280 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.541475 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.541547 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 10:52:33.541733 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.538336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.39.130:54273 (local address 127.9.39.131:43329)
0927 10:52:33.538466 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.538471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.538486 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:52:33.538576 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.538579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.538630 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.538741 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 10:52:33.539033 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.539592 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.540730 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.541145 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.541289 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.541340 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.541540 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.541613 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 10:52:33.541703 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.538021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:34601 (local address 127.9.39.130:33313)
0927 10:52:33.538414 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.538429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:52:33.538554 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:52:33.538896 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:52:33.538899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:52:33.538916 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 10:52:33.539038 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.539043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.539803 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.539805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.540602 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:52:33.540610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.541239 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.541246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:52:33.541336 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.541369 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 10:52:33.541439 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 10:52:33.541535 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":6,"thread_start_us":143,"threads_started":1}
I20260927 10:52:33.542352 29508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2e059ed1434e918368dc98bc4ae535" candidate_uuid: "568075cd551c4561a62489f12e17d61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad7731e6de7b4d8d9f6625922342af1f" is_pre_election: true
I20260927 10:52:33.542659 29508 raft_consensus.cc:2454] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 568075cd551c4561a62489f12e17d61c in term 0.
I20260927 10:52:33.542618 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2e059ed1434e918368dc98bc4ae535" candidate_uuid: "568075cd551c4561a62489f12e17d61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" is_pre_election: true
I20260927 10:52:33.542909 29246 raft_consensus.cc:2454] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 568075cd551c4561a62489f12e17d61c in term 0.
I20260927 10:52:33.543213 29313 leader_election.cc:302] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 568075cd551c4561a62489f12e17d61c, ad7731e6de7b4d8d9f6625922342af1f; no voters:
I20260927 10:52:33.543494 29578 raft_consensus.cc:2781] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 10:52:33.543581 29578 raft_consensus.cc:482] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 10:52:33.543628 29578 raft_consensus.cc:3037] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:33.544641 29578 raft_consensus.cc:504] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.544998 29578 leader_election.cc:288] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [CANDIDATE]: Term 1 election: Requested vote from peers ad7731e6de7b4d8d9f6625922342af1f (127.9.39.131:43329), 5dcd958be02a40e1bb5fddc245da8ab9 (127.9.39.129:34601)
I20260927 10:52:33.545341 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2e059ed1434e918368dc98bc4ae535" candidate_uuid: "568075cd551c4561a62489f12e17d61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dcd958be02a40e1bb5fddc245da8ab9"
I20260927 10:52:33.545465 29246 raft_consensus.cc:3037] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:33.545534 29508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2e059ed1434e918368dc98bc4ae535" candidate_uuid: "568075cd551c4561a62489f12e17d61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad7731e6de7b4d8d9f6625922342af1f"
I20260927 10:52:33.545676 29508 raft_consensus.cc:3037] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:33.546643 29246 raft_consensus.cc:2454] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 568075cd551c4561a62489f12e17d61c in term 1.
I20260927 10:52:33.546847 29508 raft_consensus.cc:2454] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 568075cd551c4561a62489f12e17d61c in term 1.
I20260927 10:52:33.547019 29310 leader_election.cc:302] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 568075cd551c4561a62489f12e17d61c, 5dcd958be02a40e1bb5fddc245da8ab9; no voters:
I20260927 10:52:33.547216 29578 raft_consensus.cc:2781] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:33.547437 29578 raft_consensus.cc:686] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 1 LEADER]: Becoming Leader. State: Replica: 568075cd551c4561a62489f12e17d61c, State: Running, Role: LEADER
I20260927 10:52:33.547758 29578 consensus_queue.cc:237] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } }
I20260927 10:52:33.550863 29103 catalog_manager.cc:5495] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c reported cstate change: term changed from 0 to 1, leader changed from <none> to 568075cd551c4561a62489f12e17d61c (127.9.39.130). New cstate: current_term: 1 leader_uuid: "568075cd551c4561a62489f12e17d61c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "568075cd551c4561a62489f12e17d61c" member_type: VOTER last_known_addr { host: "127.9.39.130" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 } health_report { overall_health: UNKNOWN } } }
I20260927 10:52:33.583981 29569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.580329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.9.39.130:43939)
0927 10:52:33.580468 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:33.580476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:33.580552 (+ 76us) server_negotiation.cc:407] Connection header received
0927 10:52:33.580743 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:33.580745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:33.580804 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:33.580891 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:52:33.581107 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.581675 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.582597 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:33.582860 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:33.582938 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:33.583239 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:52:33.583637 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:52:33.583739 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:33.583818 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 10:52:33.583860 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:52:33.589534 29246 raft_consensus.cc:1261] T 1f2e059ed1434e918368dc98bc4ae535 P 5dcd958be02a40e1bb5fddc245da8ab9 [term 1 FOLLOWER]: Refusing update from remote peer 568075cd551c4561a62489f12e17d61c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:33.589533 29508 raft_consensus.cc:1261] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 1 FOLLOWER]: Refusing update from remote peer 568075cd551c4561a62489f12e17d61c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:52:33.590245 29581 consensus_queue.cc:1035] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dcd958be02a40e1bb5fddc245da8ab9" member_type: VOTER last_known_addr { host: "127.9.39.129" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:33.590420 29578 consensus_queue.cc:1035] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad7731e6de7b4d8d9f6625922342af1f" member_type: VOTER last_known_addr { host: "127.9.39.131" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:52:33.605967 29104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:33.606150 29104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 10:52:33.608583 29104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6554093d66145db95bcf9511f2f862f: Sending DeleteTablet for 3 replicas of tablet 1f2e059ed1434e918368dc98bc4ae535
I20260927 10:52:33.609290 29226 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2e059ed1434e918368dc98bc4ae535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:33 UTC) from {username='slave'} at 127.0.0.1:36248
I20260927 10:52:33.609289 29357 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2e059ed1434e918368dc98bc4ae535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:33 UTC) from {username='slave'} at 127.0.0.1:45840
I20260927 10:52:33.609289 29488 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2e059ed1434e918368dc98bc4ae535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 10:52:33 UTC) from {username='slave'} at 127.0.0.1:40152
I20260927 10:52:33.609758 29595 tablet_replica.cc:330] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: stopping tablet replica
I20260927 10:52:33.609913 29594 tablet_replica.cc:330] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: stopping tablet replica
I20260927 10:52:33.610042 29595 raft_consensus.cc:2229] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 1 LEADER]: Raft consensus shutting down.
I20260927 10:52:33.610275 29594 raft_consensus.cc:2229] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 10:52:33.610371 29595 raft_consensus.cc:2258] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:33.610574 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29164
I20260927 10:52:33.611480 29594 raft_consensus.cc:2258] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 10:52:33.613126 29594 ts_tablet_manager.cc:1905] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:33.614339 29595 ts_tablet_manager.cc:1905] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 10:52:33.616057 29594 ts_tablet_manager.cc:1918] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:33.616240 29594 log.cc:1196] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-2/wal/wals/1f2e059ed1434e918368dc98bc4ae535
I20260927 10:52:33.616683 29594 ts_tablet_manager.cc:1939] T 1f2e059ed1434e918368dc98bc4ae535 P ad7731e6de7b4d8d9f6625922342af1f: Deleting consensus metadata
I20260927 10:52:33.617774 29595 ts_tablet_manager.cc:1918] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 10:52:33.617871 29595 log.cc:1196] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Deleting WAL directory at /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790506104903010-9374-0/minicluster-data/ts-1/wal/wals/1f2e059ed1434e918368dc98bc4ae535
I20260927 10:52:33.618013 29091 catalog_manager.cc:4841] TS ad7731e6de7b4d8d9f6625922342af1f (127.9.39.131:43329): tablet 1f2e059ed1434e918368dc98bc4ae535 (table test-table [id=671c7001e37341f2be5151d1cd96ac3c]) successfully deleted
I20260927 10:52:33.618173 29595 ts_tablet_manager.cc:1939] T 1f2e059ed1434e918368dc98bc4ae535 P 568075cd551c4561a62489f12e17d61c: Deleting consensus metadata
W20260927 10:52:33.618677 29088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.39.129:34601 (error 108)
I20260927 10:52:33.619297 29089 catalog_manager.cc:4841] TS 568075cd551c4561a62489f12e17d61c (127.9.39.130:43939): tablet 1f2e059ed1434e918368dc98bc4ae535 (table test-table [id=671c7001e37341f2be5151d1cd96ac3c]) successfully deleted
I20260927 10:52:33.619714 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:33.619422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.129:34601 (local address 127.0.0.1:36264)
0927 10:52:33.619585 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 10:52:33.619643 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260927 10:52:33.619859 29088 catalog_manager.cc:4584] TS 5dcd958be02a40e1bb5fddc245da8ab9 (127.9.39.129:34601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f2e059ed1434e918368dc98bc4ae535: Network error: Client connection negotiation failed: client connection to 127.9.39.129:34601: connect: Connection refused (error 111)
I20260927 10:52:33.620110 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29295
I20260927 10:52:33.625378 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29426
I20260927 10:52:33.630112 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29073
2026-09-27T10:52:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1067 ms)
[----------] 6 tests from EncryptionPolicyTest (9957 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:33 dist-test-slave-p9q6 krb5kdc[29599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:33 dist-test-slave-p9q6 krb5kdc[29599](info): set up 2 sockets
Sep 27 10:52:33 dist-test-slave-p9q6 krb5kdc[29599](info): commencing operation
krb5kdc: starting...
W20260927 10:52:34.670884 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:34 dist-test-slave-p9q6 krb5kdc[29599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:34.793452 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:34.900166 29612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:34.900439 29612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:34.900481 29612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:34.904572 29612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:34.904633 29612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:34.904686 29612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:34.904726 29612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:34.904753 29612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:34.910140 29612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:34.911257 29612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:34.912245 29612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:34.920397 29617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:34.920426 29618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:34.920398 29620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:34.920786 29612 server_base.cc:1034] running on GCE node
I20260927 10:52:34.921353 29612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:34.922358 29612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:34.923537 29612 hybrid_clock.cc:648] HybridClock initialized: now 1790506354923526 us; error 31 us; skew 500 ppm
Sep 27 10:52:34 dist-test-slave-p9q6 krb5kdc[29599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506354, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:34.926178 29612 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:34.927083 29612 webserver.cc:460] Webserver started at http://127.0.0.1:46239/ using document root <none> and password file <none>
I20260927 10:52:34.927619 29612 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:34.927706 29612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:34.927946 29612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:34.929883 29612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "4c1eac277f6a4177a2263d38da4ca719"
format_stamp: "Formatted at 2026-09-27 10:52:34 on dist-test-slave-p9q6"
server_key: "235558100fc67daa0765d91b347e556d"
server_key_iv: "3eb5ce64a98360f1a3e30b69ca48623c"
server_key_version: "encryptionkey@0"
I20260927 10:52:34.930424 29612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "4c1eac277f6a4177a2263d38da4ca719"
format_stamp: "Formatted at 2026-09-27 10:52:34 on dist-test-slave-p9q6"
server_key: "235558100fc67daa0765d91b347e556d"
server_key_iv: "3eb5ce64a98360f1a3e30b69ca48623c"
server_key_version: "encryptionkey@0"
I20260927 10:52:34.934015 29612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:52:34.936425 29627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:34.937546 29612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 10:52:34.937683 29612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "4c1eac277f6a4177a2263d38da4ca719"
format_stamp: "Formatted at 2026-09-27 10:52:34 on dist-test-slave-p9q6"
server_key: "235558100fc67daa0765d91b347e556d"
server_key_iv: "3eb5ce64a98360f1a3e30b69ca48623c"
server_key_version: "encryptionkey@0"
I20260927 10:52:34.937798 29612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:34.953375 29612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:34.956660 29612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:34.956872 29612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:34.963923 29612 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39071
I20260927 10:52:34.963948 29679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39071 every 8 connection(s)
I20260927 10:52:34.964865 29612 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:34.967519 29680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:34.969332 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29612
I20260927 10:52:34.969489 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:34.969800 9374 external_mini_cluster.cc:1428] Setting key 097f723a25ec57802d4ff3311e547f47
I20260927 10:52:34.973769 29680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719: Bootstrap starting.
I20260927 10:52:34.975927 29680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:34.976815 29680 log.cc:824] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719: Log is configured to *not* fsync() on all Append() calls
Sep 27 10:52:34 dist-test-slave-p9q6 krb5kdc[29599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 10:52:34.978737 29680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719: No bootstrap required, opened a new log
I20260927 10:52:34.981933 29680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1eac277f6a4177a2263d38da4ca719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260927 10:52:34.982165 29680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:34.982255 29680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1eac277f6a4177a2263d38da4ca719, State: Initialized, Role: FOLLOWER
I20260927 10:52:34.982771 29680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1eac277f6a4177a2263d38da4ca719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260927 10:52:34.982924 29680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:34.982988 29680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:34.983085 29680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:34.983601 29683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:34.971150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35177 (local address 127.0.0.1:39071)
0927 10:52:34.971548 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:34.971563 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:34.971597 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:52:34.972215 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:34.972230 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:34.972545 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:34.972884 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:34.973646 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:34.974643 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:34.975533 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:34.975991 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:34.976162 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:34.978776 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:34.978781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:34.978798 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:34.978849 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:34.980536 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:34.981036 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:34.981040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:34.981045 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:34.981147 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:34.981481 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:34.981490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:34.981492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:34.981843 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:34.982073 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:34.982336 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 10:52:34.982677 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":125,"threads_started":1}
I20260927 10:52:34.984007 29680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1eac277f6a4177a2263d38da4ca719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260927 10:52:34.984340 29680 leader_election.cc:302] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c1eac277f6a4177a2263d38da4ca719; no voters:
I20260927 10:52:34.984638 29680 leader_election.cc:288] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:34.984762 29685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:34.984956 29685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [term 1 LEADER]: Becoming Leader. State: Replica: 4c1eac277f6a4177a2263d38da4ca719, State: Running, Role: LEADER
I20260927 10:52:34.985293 29685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1eac277f6a4177a2263d38da4ca719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260927 10:52:34.985837 29680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:34.986506 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:34.986488 29685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c1eac277f6a4177a2263d38da4ca719. Latest consensus state: current_term: 1 leader_uuid: "4c1eac277f6a4177a2263d38da4ca719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1eac277f6a4177a2263d38da4ca719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260927 10:52:34.986589 29685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:34.986955 29691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:34.988849 29686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c1eac277f6a4177a2263d38da4ca719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1eac277f6a4177a2263d38da4ca719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260927 10:52:34.989104 29686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:34.989921 29691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:34.995644 29691 catalog_manager.cc:1347] Generated new cluster ID: 31335ca564d34bd7bc6dc80ab5bbad08
I20260927 10:52:34.995702 29691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:34.999001 29683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:34.988939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49585 (local address 127.0.0.1:39071)
0927 10:52:34.989073 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:34.989078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:34.990971 (+ 1893us) server_negotiation.cc:407] Connection header received
0927 10:52:34.991082 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:34.991085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:34.991135 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:34.991228 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:34.992042 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:34.992547 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:34.993346 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:34.993597 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:34.993713 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:34.995438 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:34.995442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:34.995445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:34.995469 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:34.997132 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:34.997657 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:34.997660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:34.997662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:34.997749 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:34.998070 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:34.998074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:34.998075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:34.998278 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:34.998439 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:34.998680 (+ 241us) server_negotiation.cc:299] Negotiation successful
0927 10:52:34.998833 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 10:52:35.002209 29691 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:35.003113 29691 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:35.012373 29691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c1eac277f6a4177a2263d38da4ca719: Generated new TSK 0
I20260927 10:52:35.013157 29691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:35.018601 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29612
W20260927 10:52:35.023998 29597 connection.cc:535] client connection to 127.0.0.1:39071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39071): Connection reset by peer (error 104)
2026-09-27T10:52:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:35 dist-test-slave-p9q6 krb5kdc[29711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:35 dist-test-slave-p9q6 krb5kdc[29711](info): set up 2 sockets
Sep 27 10:52:35 dist-test-slave-p9q6 krb5kdc[29711](info): commencing operation
krb5kdc: starting...
W20260927 10:52:36.038941 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:36 dist-test-slave-p9q6 krb5kdc[29711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:36.162153 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:36.269860 29724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:36.270149 29724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:36.270228 29724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:36.274158 29724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:36.274250 29724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:36.274286 29724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:36.274345 29724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:36.274384 29724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:36.279737 29724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:36.280754 29724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:36.281737 29724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:36.290010 29732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:36.290086 29724 server_base.cc:1034] running on GCE node
W20260927 10:52:36.290023 29730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:36.290026 29729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:36.290774 29724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:36.291847 29724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:36.293037 29724 hybrid_clock.cc:648] HybridClock initialized: now 1790506356293028 us; error 30 us; skew 500 ppm
Sep 27 10:52:36 dist-test-slave-p9q6 krb5kdc[29711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:36.295789 29724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:36.296567 29724 webserver.cc:460] Webserver started at http://127.0.0.1:42265/ using document root <none> and password file <none>
I20260927 10:52:36.297052 29724 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:36.297107 29724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:36.297289 29724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:36.299256 29724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "e47c8d60dd1145ea99b5cca28114282f"
format_stamp: "Formatted at 2026-09-27 10:52:36 on dist-test-slave-p9q6"
server_key: "d180f4e7b7446eb6268a21eabcb9d80a"
server_key_iv: "bb38245d9af3d56c16e9465f8ee4ac10"
server_key_version: "encryptionkey@0"
I20260927 10:52:36.299734 29724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "e47c8d60dd1145ea99b5cca28114282f"
format_stamp: "Formatted at 2026-09-27 10:52:36 on dist-test-slave-p9q6"
server_key: "d180f4e7b7446eb6268a21eabcb9d80a"
server_key_iv: "bb38245d9af3d56c16e9465f8ee4ac10"
server_key_version: "encryptionkey@0"
I20260927 10:52:36.303192 29724 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:52:36.305390 29739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:36.306519 29724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:36.306655 29724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "e47c8d60dd1145ea99b5cca28114282f"
format_stamp: "Formatted at 2026-09-27 10:52:36 on dist-test-slave-p9q6"
server_key: "d180f4e7b7446eb6268a21eabcb9d80a"
server_key_iv: "bb38245d9af3d56c16e9465f8ee4ac10"
server_key_version: "encryptionkey@0"
I20260927 10:52:36.306807 29724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:36.338428 29724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:36.341719 29724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:36.341933 29724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:36.349998 29724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42155
I20260927 10:52:36.350030 29791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42155 every 8 connection(s)
I20260927 10:52:36.351327 29724 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:36.354671 29792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:36.358381 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29724
I20260927 10:52:36.358558 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:36.358893 9374 external_mini_cluster.cc:1428] Setting key fbaadecd9d6e449c0ca00bc09693f220
I20260927 10:52:36.361140 29792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f: Bootstrap starting.
I20260927 10:52:36.363314 29792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:36.364033 29792 log.cc:824] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:36.366097 29792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f: No bootstrap required, opened a new log
I20260927 10:52:36.368381 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:36.360305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57513 (local address 127.0.0.1:42155)
0927 10:52:36.360724 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:36.360739 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:36.360773 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:52:36.361435 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:36.361453 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:36.361795 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:36.362113 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:36.362513 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:36.363606 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:36.364571 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:36.365176 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:36.365325 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:36.366170 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:36.366176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:36.366192 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:36.366716 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:36.366781 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:36.366811 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:36.366943 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:36.367106 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 10:52:36.367235 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":126,"threads_started":1}
I20260927 10:52:36.369743 29792 raft_consensus.cc:348] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47c8d60dd1145ea99b5cca28114282f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42155 } }
I20260927 10:52:36.370008 29792 raft_consensus.cc:374] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:36.370115 29792 raft_consensus.cc:729] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47c8d60dd1145ea99b5cca28114282f, State: Initialized, Role: FOLLOWER
I20260927 10:52:36.370599 29792 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47c8d60dd1145ea99b5cca28114282f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42155 } }
I20260927 10:52:36.370770 29792 raft_consensus.cc:388] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:36.370860 29792 raft_consensus.cc:482] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:36.370966 29792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:36.371549 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:36.372037 29792 raft_consensus.cc:504] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47c8d60dd1145ea99b5cca28114282f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42155 } }
I20260927 10:52:36.372370 29792 leader_election.cc:302] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e47c8d60dd1145ea99b5cca28114282f; no voters:
I20260927 10:52:36.372677 29792 leader_election.cc:288] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:36.372764 29798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:36.372977 29798 raft_consensus.cc:686] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [term 1 LEADER]: Becoming Leader. State: Replica: e47c8d60dd1145ea99b5cca28114282f, State: Running, Role: LEADER
I20260927 10:52:36.373310 29798 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47c8d60dd1145ea99b5cca28114282f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42155 } }
I20260927 10:52:36.373917 29792 sys_catalog.cc:564] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:36.374953 29803 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47c8d60dd1145ea99b5cca28114282f. Latest consensus state: current_term: 1 leader_uuid: "e47c8d60dd1145ea99b5cca28114282f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47c8d60dd1145ea99b5cca28114282f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42155 } } }
I20260927 10:52:36.375352 29803 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:36.375248 29802 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e47c8d60dd1145ea99b5cca28114282f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47c8d60dd1145ea99b5cca28114282f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42155 } } }
I20260927 10:52:36.375666 29802 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:36.378484 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:36.373554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60279 (local address 127.0.0.1:42155)
0927 10:52:36.373655 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:36.373658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:36.373921 (+ 263us) server_negotiation.cc:407] Connection header received
0927 10:52:36.374065 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:36.374068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:36.374108 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:36.374201 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:36.374495 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:36.375371 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:36.376734 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:36.377076 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:36.377192 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:36.378030 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:36.378033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:36.378036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:36.378047 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:36.378077 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:36.378087 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:36.378178 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:36.378318 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 10:52:36.378352 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260927 10:52:36.379201 29817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:52:36.379323 29817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:52:36.379447 29813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:36.380162 29813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:36.385481 29813 catalog_manager.cc:1347] Generated new cluster ID: b1ea24d75b144d13894a2886f17070c2
I20260927 10:52:36.385548 29813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:36.406240 29813 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:36.407549 29813 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:36.418470 29813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e47c8d60dd1145ea99b5cca28114282f: Generated new TSK 0
I20260927 10:52:36.419305 29813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:36.423678 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29724
W20260927 10:52:36.429121 29709 connection.cc:535] client connection to 127.0.0.1:42155 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42155): Connection reset by peer (error 104)
2026-09-27T10:52:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:36 dist-test-slave-p9q6 krb5kdc[29823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:36 dist-test-slave-p9q6 krb5kdc[29823](info): set up 2 sockets
Sep 27 10:52:36 dist-test-slave-p9q6 krb5kdc[29823](info): commencing operation
krb5kdc: starting...
W20260927 10:52:37.466897 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:37 dist-test-slave-p9q6 krb5kdc[29823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:37.586061 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:37.694833 29836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:37.695143 29836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:37.695218 29836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:37.699370 29836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:37.699465 29836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:37.699510 29836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:37.699566 29836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:37.699605 29836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:37.705042 29836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:37.706166 29836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:37.707219 29836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:37.715862 29844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:37.716094 29836 server_base.cc:1034] running on GCE node
W20260927 10:52:37.715829 29841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:37.715839 29842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:37.716758 29836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:37.717801 29836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:37.718952 29836 hybrid_clock.cc:648] HybridClock initialized: now 1790506357718940 us; error 31 us; skew 500 ppm
Sep 27 10:52:37 dist-test-slave-p9q6 krb5kdc[29823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:37.726675 29836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:37.727592 29836 webserver.cc:460] Webserver started at http://127.0.0.1:42173/ using document root <none> and password file <none>
I20260927 10:52:37.728149 29836 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:37.728235 29836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:37.728458 29836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:37.730453 29836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "f63b13fa31a44221b5c80f8c994eb946"
format_stamp: "Formatted at 2026-09-27 10:52:37 on dist-test-slave-p9q6"
server_key: "53f0680461d2767405c1868dca86e264"
server_key_iv: "cd64a89a6c5e66cd64a733b73bf14267"
server_key_version: "encryptionkey@0"
I20260927 10:52:37.731086 29836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "f63b13fa31a44221b5c80f8c994eb946"
format_stamp: "Formatted at 2026-09-27 10:52:37 on dist-test-slave-p9q6"
server_key: "53f0680461d2767405c1868dca86e264"
server_key_iv: "cd64a89a6c5e66cd64a733b73bf14267"
server_key_version: "encryptionkey@0"
I20260927 10:52:37.734767 29836 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:52:37.737154 29851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:37.738407 29836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:37.738540 29836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "f63b13fa31a44221b5c80f8c994eb946"
format_stamp: "Formatted at 2026-09-27 10:52:37 on dist-test-slave-p9q6"
server_key: "53f0680461d2767405c1868dca86e264"
server_key_iv: "cd64a89a6c5e66cd64a733b73bf14267"
server_key_version: "encryptionkey@0"
I20260927 10:52:37.738660 29836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:37.743130 29836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:37.745855 29836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:37.746033 29836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:37.753240 29836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46747
I20260927 10:52:37.753266 29903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46747 every 8 connection(s)
I20260927 10:52:37.754182 29836 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:37.756783 29904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:37.762105 29904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946: Bootstrap starting.
I20260927 10:52:37.762287 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29836
I20260927 10:52:37.762460 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:37.762794 9374 external_mini_cluster.cc:1428] Setting key 79da422e4bf85c5e2febaca7e0acc84e
I20260927 10:52:37.764842 29904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:37.765692 29904 log.cc:824] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:37.767906 29904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946: No bootstrap required, opened a new log
I20260927 10:52:37.768215 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:37.764101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37685 (local address 127.0.0.1:46747)
0927 10:52:37.764563 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:37.764580 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:37.764623 (+ 43us) server_negotiation.cc:407] Connection header received
0927 10:52:37.765438 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:37.765459 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:37.765834 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:37.766193 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:37.766965 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:37.766970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:37.766983 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:37.766993 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:37.767040 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:37.767059 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:37.767141 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:37.767284 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 10:52:37.767449 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":101,"threads_started":1}
I20260927 10:52:37.770845 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:37.771806 29904 raft_consensus.cc:348] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63b13fa31a44221b5c80f8c994eb946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46747 } }
I20260927 10:52:37.772029 29904 raft_consensus.cc:374] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:37.772105 29904 raft_consensus.cc:729] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f63b13fa31a44221b5c80f8c994eb946, State: Initialized, Role: FOLLOWER
I20260927 10:52:37.772480 29904 consensus_queue.cc:260] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63b13fa31a44221b5c80f8c994eb946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46747 } }
I20260927 10:52:37.772661 29904 raft_consensus.cc:388] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:37.772758 29904 raft_consensus.cc:482] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:37.772876 29904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:37.773717 29904 raft_consensus.cc:504] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63b13fa31a44221b5c80f8c994eb946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46747 } }
I20260927 10:52:37.774075 29904 leader_election.cc:302] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f63b13fa31a44221b5c80f8c994eb946; no voters:
I20260927 10:52:37.774369 29904 leader_election.cc:288] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:37.774559 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:37.773068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56999 (local address 127.0.0.1:46747)
0927 10:52:37.773184 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:37.773189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:37.773201 (+ 12us) server_negotiation.cc:407] Connection header received
0927 10:52:37.773230 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:37.773233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:37.773261 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:37.773348 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:37.774066 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:37.774069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:37.774072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:37.774082 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:37.774107 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:37.774117 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:37.774220 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:37.774359 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 10:52:37.774423 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 10:52:37.774708 29915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:37.775398 29915 raft_consensus.cc:686] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [term 1 LEADER]: Becoming Leader. State: Replica: f63b13fa31a44221b5c80f8c994eb946, State: Running, Role: LEADER
I20260927 10:52:37.775504 29904 sys_catalog.cc:564] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:37.775699 29915 consensus_queue.cc:237] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63b13fa31a44221b5c80f8c994eb946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46747 } }
I20260927 10:52:37.777324 29914 sys_catalog.cc:455] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f63b13fa31a44221b5c80f8c994eb946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63b13fa31a44221b5c80f8c994eb946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46747 } } }
I20260927 10:52:37.777602 29914 sys_catalog.cc:458] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:37.777348 29916 sys_catalog.cc:455] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f63b13fa31a44221b5c80f8c994eb946. Latest consensus state: current_term: 1 leader_uuid: "f63b13fa31a44221b5c80f8c994eb946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63b13fa31a44221b5c80f8c994eb946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46747 } } }
I20260927 10:52:37.778007 29916 sys_catalog.cc:458] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946 [sys.catalog]: This master's current role is: LEADER
W20260927 10:52:37.780109 29928 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:52:37.780194 29928 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:52:37.780294 29929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:37.781044 29929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:37.786238 29929 catalog_manager.cc:1347] Generated new cluster ID: 3382cb373fc1465ca94e0bbe69b69e5d
I20260927 10:52:37.786298 29929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:37.793962 29929 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:37.794255 29929 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:37.798089 29929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f63b13fa31a44221b5c80f8c994eb946: Generated new TSK 0
I20260927 10:52:37.798808 29929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:37.807888 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29836
2026-09-27T10:52:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:37 dist-test-slave-p9q6 krb5kdc[29935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:37 dist-test-slave-p9q6 krb5kdc[29935](info): set up 2 sockets
Sep 27 10:52:37 dist-test-slave-p9q6 krb5kdc[29935](info): commencing operation
krb5kdc: starting...
W20260927 10:52:38.834884 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:38 dist-test-slave-p9q6 krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:38.953347 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:39.062587 29948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:39.062896 29948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:39.062969 29948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:39.066967 29948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:39.067059 29948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:39.067095 29948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:39.067153 29948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:39.067190 29948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:39.072489 29948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:39.073573 29948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:39.074611 29948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:39.083156 29954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:39.083166 29956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:39.083209 29948 server_base.cc:1034] running on GCE node
W20260927 10:52:39.083174 29953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:39.084030 29948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:39.085124 29948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:39.086331 29948 hybrid_clock.cc:648] HybridClock initialized: now 1790506359086304 us; error 44 us; skew 500 ppm
Sep 27 10:52:39 dist-test-slave-p9q6 krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506359, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:39.089123 29948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:39.090109 29948 webserver.cc:460] Webserver started at http://127.0.0.1:37133/ using document root <none> and password file <none>
I20260927 10:52:39.090679 29948 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:39.090833 29948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:39.091070 29948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:39.093109 29948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "eb3eb113b7784067a747f3888d87ac2b"
format_stamp: "Formatted at 2026-09-27 10:52:39 on dist-test-slave-p9q6"
server_key: "58bf6d583b3a369caf22c1a58ffcacd6"
server_key_iv: "35d9e1450d02d1324f2555b189dcae22"
server_key_version: "encryptionkey@0"
I20260927 10:52:39.093663 29948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "eb3eb113b7784067a747f3888d87ac2b"
format_stamp: "Formatted at 2026-09-27 10:52:39 on dist-test-slave-p9q6"
server_key: "58bf6d583b3a369caf22c1a58ffcacd6"
server_key_iv: "35d9e1450d02d1324f2555b189dcae22"
server_key_version: "encryptionkey@0"
I20260927 10:52:39.097393 29948 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260927 10:52:39.099957 29963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:39.101235 29948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:39.101403 29948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "eb3eb113b7784067a747f3888d87ac2b"
format_stamp: "Formatted at 2026-09-27 10:52:39 on dist-test-slave-p9q6"
server_key: "58bf6d583b3a369caf22c1a58ffcacd6"
server_key_iv: "35d9e1450d02d1324f2555b189dcae22"
server_key_version: "encryptionkey@0"
I20260927 10:52:39.101559 29948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:39.121243 29948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:39.124205 29948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:39.124408 29948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:39.131547 29948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41143
I20260927 10:52:39.131572 30015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41143 every 8 connection(s)
I20260927 10:52:39.132531 29948 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:39.135229 30016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:39.139533 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 29948
I20260927 10:52:39.139678 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:39.139945 9374 external_mini_cluster.cc:1428] Setting key 7295477211101cb68508eb8fa5d686fc
I20260927 10:52:39.140642 30016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b: Bootstrap starting.
I20260927 10:52:39.142952 30016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:39.143874 30016 log.cc:824] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:39.145876 30016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b: No bootstrap required, opened a new log
Sep 27 10:52:39 dist-test-slave-p9q6 krb5kdc[29935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 10:52:39.150085 30016 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3eb113b7784067a747f3888d87ac2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260927 10:52:39.150334 30016 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:39.150410 30016 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb3eb113b7784067a747f3888d87ac2b, State: Initialized, Role: FOLLOWER
I20260927 10:52:39.150943 30016 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3eb113b7784067a747f3888d87ac2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260927 10:52:39.151129 30016 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:39.151201 30016 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:39.151301 30016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:39.152356 30016 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3eb113b7784067a747f3888d87ac2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260927 10:52:39.152732 30016 leader_election.cc:302] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb3eb113b7784067a747f3888d87ac2b; no voters:
I20260927 10:52:39.153056 30016 leader_election.cc:288] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:39.153157 30021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:39.153406 30021 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [term 1 LEADER]: Becoming Leader. State: Replica: eb3eb113b7784067a747f3888d87ac2b, State: Running, Role: LEADER
I20260927 10:52:39.153954 30021 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3eb113b7784067a747f3888d87ac2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260927 10:52:39.154464 30016 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:39.154784 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:39.141283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46077 (local address 127.0.0.1:41143)
0927 10:52:39.141679 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:39.141694 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:39.141728 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:52:39.142325 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:39.142341 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:39.142682 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:39.143061 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:39.143932 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:39.145001 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:39.146127 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:39.146589 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:39.146781 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:39.149489 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:39.149494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:39.149507 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:39.149532 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:39.151560 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:39.152006 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:39.152015 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:39.152023 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:39.152172 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:39.152442 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:39.152448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:39.152450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:39.152955 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:39.153151 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:39.153536 (+ 385us) server_negotiation.cc:299] Negotiation successful
0927 10:52:39.153833 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":128,"threads_started":1}
I20260927 10:52:39.155182 30021 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb3eb113b7784067a747f3888d87ac2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3eb113b7784067a747f3888d87ac2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } } }
I20260927 10:52:39.155368 30021 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:39.155593 30022 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb3eb113b7784067a747f3888d87ac2b. Latest consensus state: current_term: 1 leader_uuid: "eb3eb113b7784067a747f3888d87ac2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3eb113b7784067a747f3888d87ac2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } } }
I20260927 10:52:39.155680 30022 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:39.155827 30030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:39.158030 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:39.158938 30030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:39.165230 30030 catalog_manager.cc:1347] Generated new cluster ID: 06e80ed4b05143948cf7df5172ebb4fa
I20260927 10:52:39.165328 30030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:39.168147 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:39.160289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33187 (local address 127.0.0.1:41143)
0927 10:52:39.160412 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:39.160417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:39.160687 (+ 270us) server_negotiation.cc:407] Connection header received
0927 10:52:39.160861 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:39.160863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:39.160908 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:39.160962 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:39.161670 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:39.162178 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:39.163138 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:39.163428 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:39.163489 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:39.164981 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:39.164985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:39.164988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:39.165010 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:39.166455 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:39.166942 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:39.166946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:39.166948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:39.167028 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:39.167310 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:39.167314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:39.167316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:39.167487 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:39.167632 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:39.167853 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 10:52:39.167968 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:52:39.176852 30030 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:39.178051 30030 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:39.185035 30030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb3eb113b7784067a747f3888d87ac2b: Generated new TSK 0
I20260927 10:52:39.185812 30030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:39.193347 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 29948
2026-09-27T10:52:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:39 dist-test-slave-p9q6 krb5kdc[30047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:39 dist-test-slave-p9q6 krb5kdc[30047](info): set up 2 sockets
Sep 27 10:52:39 dist-test-slave-p9q6 krb5kdc[30047](info): commencing operation
krb5kdc: starting...
W20260927 10:52:40.231091 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:40 dist-test-slave-p9q6 krb5kdc[30047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:40.358628 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:40.474869 30060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:40.475165 30060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:40.475210 30060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:40.479334 30060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:40.479390 30060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:40.479416 30060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:40.479432 30060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:40.479447 30060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:40.485809 30060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:40.487048 30060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:40.488062 30060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:40.496882 30065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:40.496925 30066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:40.496892 30068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:40.497473 30060 server_base.cc:1034] running on GCE node
I20260927 10:52:40.498198 30060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:40.499543 30060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:40.500720 30060 hybrid_clock.cc:648] HybridClock initialized: now 1790506360500691 us; error 48 us; skew 500 ppm
Sep 27 10:52:40 dist-test-slave-p9q6 krb5kdc[30047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:40.503556 30060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:40.504498 30060 webserver.cc:460] Webserver started at http://127.0.0.1:37621/ using document root <none> and password file <none>
I20260927 10:52:40.505061 30060 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:40.505129 30060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:40.505291 30060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:40.507334 30060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "a81451d978c04d3cb85396a8b3f83952"
format_stamp: "Formatted at 2026-09-27 10:52:40 on dist-test-slave-p9q6"
server_key: "fc3a92b37709fe6717424c2f8028aa29"
server_key_iv: "d6c8a783646fd1175b900937f2b9a6fe"
server_key_version: "encryptionkey@0"
I20260927 10:52:40.507836 30060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "a81451d978c04d3cb85396a8b3f83952"
format_stamp: "Formatted at 2026-09-27 10:52:40 on dist-test-slave-p9q6"
server_key: "fc3a92b37709fe6717424c2f8028aa29"
server_key_iv: "d6c8a783646fd1175b900937f2b9a6fe"
server_key_version: "encryptionkey@0"
I20260927 10:52:40.511554 30060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:52:40.513890 30075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:40.515152 30060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:40.515275 30060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "a81451d978c04d3cb85396a8b3f83952"
format_stamp: "Formatted at 2026-09-27 10:52:40 on dist-test-slave-p9q6"
server_key: "fc3a92b37709fe6717424c2f8028aa29"
server_key_iv: "d6c8a783646fd1175b900937f2b9a6fe"
server_key_version: "encryptionkey@0"
I20260927 10:52:40.515380 30060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:40.528784 30060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:40.532264 30060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:40.532480 30060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:40.539974 30060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38765
I20260927 10:52:40.540002 30127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38765 every 8 connection(s)
I20260927 10:52:40.540925 30060 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:40.543635 30128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:40.549168 30128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952: Bootstrap starting.
I20260927 10:52:40.549777 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30060
I20260927 10:52:40.549921 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:40.550205 9374 external_mini_cluster.cc:1428] Setting key d610b8995d23d44d3d686605aa028003
I20260927 10:52:40.552003 30128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:40.552728 30128 log.cc:824] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:40.554833 30128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952: No bootstrap required, opened a new log
I20260927 10:52:40.558459 30128 raft_consensus.cc:348] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81451d978c04d3cb85396a8b3f83952" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38765 } }
I20260927 10:52:40.558736 30128 raft_consensus.cc:374] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:40.558799 30128 raft_consensus.cc:729] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a81451d978c04d3cb85396a8b3f83952, State: Initialized, Role: FOLLOWER
I20260927 10:52:40.559100 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:40.551487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59671 (local address 127.0.0.1:38765)
0927 10:52:40.551906 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:40.551921 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:40.551953 (+ 32us) server_negotiation.cc:407] Connection header received
0927 10:52:40.552629 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:40.552647 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:40.553043 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:40.553431 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:40.553808 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:40.554867 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:40.555774 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:40.556247 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:40.556412 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:40.557294 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:40.557299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:40.557310 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:40.557682 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:40.557735 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:40.557757 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:40.557863 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:40.558011 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 10:52:40.558220 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":135,"threads_started":1}
I20260927 10:52:40.559338 30128 consensus_queue.cc:260] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81451d978c04d3cb85396a8b3f83952" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38765 } }
I20260927 10:52:40.559506 30128 raft_consensus.cc:388] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:40.559573 30128 raft_consensus.cc:482] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:40.559669 30128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:40.560539 30128 raft_consensus.cc:504] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81451d978c04d3cb85396a8b3f83952" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38765 } }
I20260927 10:52:40.560914 30128 leader_election.cc:302] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a81451d978c04d3cb85396a8b3f83952; no voters:
I20260927 10:52:40.561278 30128 leader_election.cc:288] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:40.561376 30133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:40.561609 30133 raft_consensus.cc:686] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [term 1 LEADER]: Becoming Leader. State: Replica: a81451d978c04d3cb85396a8b3f83952, State: Running, Role: LEADER
I20260927 10:52:40.561959 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:40.561924 30133 consensus_queue.cc:237] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81451d978c04d3cb85396a8b3f83952" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38765 } }
I20260927 10:52:40.562561 30128 sys_catalog.cc:564] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:40.563351 30135 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a81451d978c04d3cb85396a8b3f83952. Latest consensus state: current_term: 1 leader_uuid: "a81451d978c04d3cb85396a8b3f83952" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81451d978c04d3cb85396a8b3f83952" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38765 } } }
I20260927 10:52:40.563476 30135 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:40.563798 30134 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a81451d978c04d3cb85396a8b3f83952" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81451d978c04d3cb85396a8b3f83952" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38765 } } }
I20260927 10:52:40.563889 30134 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:40.564924 30146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:40.568066 30146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:40.569108 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:40.564393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52083 (local address 127.0.0.1:38765)
0927 10:52:40.564485 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:40.564489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:40.565026 (+ 537us) server_negotiation.cc:407] Connection header received
0927 10:52:40.565194 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:40.565198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:40.565251 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:40.565326 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:40.565683 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:40.566195 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:40.567341 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:40.567674 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:40.567775 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:40.568654 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:40.568658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:40.568661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:40.568673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:40.568701 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:40.568716 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:40.568794 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:40.568923 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 10:52:40.568976 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 10:52:40.574205 30146 catalog_manager.cc:1347] Generated new cluster ID: b750bc5629134ac09d3d1b25ed51cc1d
I20260927 10:52:40.574285 30146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:40.587239 30146 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:40.588253 30146 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:40.598770 30146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a81451d978c04d3cb85396a8b3f83952: Generated new TSK 0
I20260927 10:52:40.599491 30146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:40.609299 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30060
2026-09-27T10:52:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:40 dist-test-slave-p9q6 krb5kdc[30159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:40 dist-test-slave-p9q6 krb5kdc[30159](info): set up 2 sockets
Sep 27 10:52:40 dist-test-slave-p9q6 krb5kdc[30159](info): commencing operation
krb5kdc: starting...
W20260927 10:52:41.634910 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:41 dist-test-slave-p9q6 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:41.754165 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:41.878746 30172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:41.879041 30172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:41.879164 30172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:41.883167 30172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:41.883240 30172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:41.883275 30172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:41.883328 30172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:41.883363 30172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:41.888414 30172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:41.889439 30172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:41.890578 30172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:41.899374 30177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:41.899374 30180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:41.899411 30178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:41.899695 30172 server_base.cc:1034] running on GCE node
I20260927 10:52:41.900241 30172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:41.901358 30172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:41.902556 30172 hybrid_clock.cc:648] HybridClock initialized: now 1790506361902530 us; error 40 us; skew 500 ppm
Sep 27 10:52:41 dist-test-slave-p9q6 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506361, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:41.905296 30172 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:41.906181 30172 webserver.cc:460] Webserver started at http://127.0.0.1:44841/ using document root <none> and password file <none>
I20260927 10:52:41.906780 30172 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:41.906870 30172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:41.907104 30172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:41.909076 30172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "2f0a4a55fab549579f52be6fbdf493e3"
format_stamp: "Formatted at 2026-09-27 10:52:41 on dist-test-slave-p9q6"
server_key: "0dc3e58af7e28de6cb91fdfec4edc373"
server_key_iv: "f428ce129b0254c498b260eccdcc8af9"
server_key_version: "encryptionkey@0"
I20260927 10:52:41.909865 30172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "2f0a4a55fab549579f52be6fbdf493e3"
format_stamp: "Formatted at 2026-09-27 10:52:41 on dist-test-slave-p9q6"
server_key: "0dc3e58af7e28de6cb91fdfec4edc373"
server_key_iv: "f428ce129b0254c498b260eccdcc8af9"
server_key_version: "encryptionkey@0"
I20260927 10:52:41.913905 30172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 10:52:41.916524 30187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:41.917831 30172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:41.917986 30172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "2f0a4a55fab549579f52be6fbdf493e3"
format_stamp: "Formatted at 2026-09-27 10:52:41 on dist-test-slave-p9q6"
server_key: "0dc3e58af7e28de6cb91fdfec4edc373"
server_key_iv: "f428ce129b0254c498b260eccdcc8af9"
server_key_version: "encryptionkey@0"
I20260927 10:52:41.918109 30172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:41.923110 30172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:41.926577 30172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:41.926800 30172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:41.934514 30172 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36881
I20260927 10:52:41.934520 30239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36881 every 8 connection(s)
I20260927 10:52:41.935532 30172 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:41.938184 30240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:41.942253 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30172
I20260927 10:52:41.942466 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:41.942835 9374 external_mini_cluster.cc:1428] Setting key 27e9cfa0ddc8a7cce1bbd7d4eec7e959
I20260927 10:52:41.943944 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3: Bootstrap starting.
I20260927 10:52:41.946136 30240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:41.947062 30240 log.cc:824] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:41.948482 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:41.944314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48631 (local address 127.0.0.1:36881)
0927 10:52:41.944748 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:41.944772 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:41.944810 (+ 38us) server_negotiation.cc:407] Connection header received
0927 10:52:41.945497 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:41.945516 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:41.945819 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:41.946128 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:41.947017 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:41.947022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:41.947034 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:41.947044 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:41.947105 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:41.947125 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:41.947288 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:41.947469 (+ 181us) server_negotiation.cc:299] Negotiation successful
0927 10:52:41.947649 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":140,"threads_started":1}
I20260927 10:52:41.949136 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3: No bootstrap required, opened a new log
I20260927 10:52:41.951447 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:41.952912 30240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a4a55fab549579f52be6fbdf493e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260927 10:52:41.953155 30240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:41.953241 30240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0a4a55fab549579f52be6fbdf493e3, State: Initialized, Role: FOLLOWER
I20260927 10:52:41.953702 30240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a4a55fab549579f52be6fbdf493e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260927 10:52:41.953895 30240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:41.953974 30240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:41.954110 30240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:41.955116 30240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a4a55fab549579f52be6fbdf493e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260927 10:52:41.955334 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:41.953549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49545 (local address 127.0.0.1:36881)
0927 10:52:41.953675 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:41.953678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:41.953876 (+ 198us) server_negotiation.cc:407] Connection header received
0927 10:52:41.954046 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:41.954049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:41.954083 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:41.954162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:41.954956 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:41.954963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:41.954969 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:41.954984 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:41.955014 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:41.955029 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:41.955124 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:41.955192 (+ 68us) server_negotiation.cc:299] Negotiation successful
0927 10:52:41.955231 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 10:52:41.955490 30240 leader_election.cc:302] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f0a4a55fab549579f52be6fbdf493e3; no voters:
I20260927 10:52:41.955786 30240 leader_election.cc:288] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:41.955912 30250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:41.956130 30250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [term 1 LEADER]: Becoming Leader. State: Replica: 2f0a4a55fab549579f52be6fbdf493e3, State: Running, Role: LEADER
I20260927 10:52:41.956470 30250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a4a55fab549579f52be6fbdf493e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260927 10:52:41.957000 30240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:41.958227 30252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f0a4a55fab549579f52be6fbdf493e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a4a55fab549579f52be6fbdf493e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } } }
I20260927 10:52:41.958266 30251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0a4a55fab549579f52be6fbdf493e3. Latest consensus state: current_term: 1 leader_uuid: "2f0a4a55fab549579f52be6fbdf493e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a4a55fab549579f52be6fbdf493e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } } }
I20260927 10:52:41.958356 30252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:41.958374 30251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:41.958750 30259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:41.962144 30259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:41.968273 30259 catalog_manager.cc:1347] Generated new cluster ID: 250c3b4a705f4edab228819df1ebedcc
I20260927 10:52:41.968371 30259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:41.987769 30259 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:41.988091 30259 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:41.993517 30259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f0a4a55fab549579f52be6fbdf493e3: Generated new TSK 0
I20260927 10:52:41.994153 30259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:42.003815 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30172
2026-09-27T10:52:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:42 dist-test-slave-p9q6 krb5kdc[30271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:42 dist-test-slave-p9q6 krb5kdc[30271](info): set up 2 sockets
Sep 27 10:52:42 dist-test-slave-p9q6 krb5kdc[30271](info): commencing operation
krb5kdc: starting...
W20260927 10:52:43.035012 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:43 dist-test-slave-p9q6 krb5kdc[30271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:43.154124 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:43.261456 30284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:43.261765 30284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:43.261845 30284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:43.265868 30284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:43.265964 30284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:43.266008 30284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:43.266068 30284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:43.266108 30284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:43.271245 30284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:43.272289 30284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:43.273294 30284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:43.281388 30292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:43.281373 30290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:43.281388 30289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:43.281659 30284 server_base.cc:1034] running on GCE node
I20260927 10:52:43.282264 30284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:43.283375 30284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:43.284567 30284 hybrid_clock.cc:648] HybridClock initialized: now 1790506363284558 us; error 27 us; skew 500 ppm
Sep 27 10:52:43 dist-test-slave-p9q6 krb5kdc[30271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506363, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:43.293852 30284 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:43.294871 30284 webserver.cc:460] Webserver started at http://127.0.0.1:40791/ using document root <none> and password file <none>
I20260927 10:52:43.295478 30284 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:43.295573 30284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:43.295804 30284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:43.297870 30284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "348d6e7a0ab145bab1774cf9221b2096"
format_stamp: "Formatted at 2026-09-27 10:52:43 on dist-test-slave-p9q6"
server_key: "89b0a866267ef8b451ee91f4b753c726"
server_key_iv: "0f920f9940a4b883c1fbc0c6927e90ff"
server_key_version: "encryptionkey@0"
I20260927 10:52:43.298448 30284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "348d6e7a0ab145bab1774cf9221b2096"
format_stamp: "Formatted at 2026-09-27 10:52:43 on dist-test-slave-p9q6"
server_key: "89b0a866267ef8b451ee91f4b753c726"
server_key_iv: "0f920f9940a4b883c1fbc0c6927e90ff"
server_key_version: "encryptionkey@0"
I20260927 10:52:43.302177 30284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:52:43.304664 30299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:43.305905 30284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:43.306047 30284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "348d6e7a0ab145bab1774cf9221b2096"
format_stamp: "Formatted at 2026-09-27 10:52:43 on dist-test-slave-p9q6"
server_key: "89b0a866267ef8b451ee91f4b753c726"
server_key_iv: "0f920f9940a4b883c1fbc0c6927e90ff"
server_key_version: "encryptionkey@0"
I20260927 10:52:43.306161 30284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:43.326437 30284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:43.329447 30284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:43.329649 30284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:43.336597 30284 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45627
I20260927 10:52:43.336620 30351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45627 every 8 connection(s)
I20260927 10:52:43.337525 30284 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:43.339980 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30284
I20260927 10:52:43.340224 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:43.340312 30352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:43.340574 9374 external_mini_cluster.cc:1428] Setting key a39a824c0c54d29e7bc4bbde9d79ed0c
I20260927 10:52:43.346424 30352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096: Bootstrap starting.
I20260927 10:52:43.348690 30352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096: Neither blocks nor log segments found. Creating new log.
Sep 27 10:52:43 dist-test-slave-p9q6 krb5kdc[30271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 10:52:43.349436 30352 log.cc:824] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:43.351400 30352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096: No bootstrap required, opened a new log
I20260927 10:52:43.354348 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:43.341942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:48305 (local address 127.0.0.1:45627)
0927 10:52:43.342393 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:43.342408 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:43.342447 (+ 39us) server_negotiation.cc:407] Connection header received
0927 10:52:43.343407 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:43.343426 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:43.343783 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:43.344136 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:43.344882 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:43.345843 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:43.346791 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:43.347251 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:43.347438 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:43.349945 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:43.349949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:43.349962 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:43.349988 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:43.351602 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:43.352111 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:43.352115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:43.352119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:43.352222 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:43.352559 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:43.352569 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:43.352571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:43.352924 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:43.353146 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:43.353453 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 10:52:43.353665 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":139,"threads_started":1}
I20260927 10:52:43.354979 30352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348d6e7a0ab145bab1774cf9221b2096" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45627 } }
I20260927 10:52:43.355216 30352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:43.355949 30352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348d6e7a0ab145bab1774cf9221b2096, State: Initialized, Role: FOLLOWER
I20260927 10:52:43.356449 30352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348d6e7a0ab145bab1774cf9221b2096" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45627 } }
I20260927 10:52:43.356604 30352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:43.356693 30352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:43.356788 30352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:43.357201 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:43.357897 30352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348d6e7a0ab145bab1774cf9221b2096" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45627 } }
I20260927 10:52:43.358263 30352 leader_election.cc:302] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 348d6e7a0ab145bab1774cf9221b2096; no voters:
I20260927 10:52:43.358593 30352 leader_election.cc:288] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:43.358769 30359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:43.359068 30359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [term 1 LEADER]: Becoming Leader. State: Replica: 348d6e7a0ab145bab1774cf9221b2096, State: Running, Role: LEADER
I20260927 10:52:43.359465 30359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348d6e7a0ab145bab1774cf9221b2096" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45627 } }
I20260927 10:52:43.359916 30352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:43.361133 30363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "348d6e7a0ab145bab1774cf9221b2096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348d6e7a0ab145bab1774cf9221b2096" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45627 } } }
I20260927 10:52:43.361266 30363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:43.361493 30363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 348d6e7a0ab145bab1774cf9221b2096. Latest consensus state: current_term: 1 leader_uuid: "348d6e7a0ab145bab1774cf9221b2096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348d6e7a0ab145bab1774cf9221b2096" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45627 } } }
I20260927 10:52:43.361574 30363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:43.361858 30372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:43.365291 30372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:43.368253 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:43.359391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:51727 (local address 127.0.0.1:45627)
0927 10:52:43.359504 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:43.359508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:43.360053 (+ 545us) server_negotiation.cc:407] Connection header received
0927 10:52:43.360123 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:43.360125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:43.360170 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:43.360253 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:43.361172 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:43.361963 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:43.363104 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:43.363323 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:43.363455 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:43.364844 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:43.364847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:43.364850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:43.364880 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:43.366365 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:43.366894 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:43.366899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:43.366906 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:43.366977 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:43.367321 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:43.367324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:43.367326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:43.367546 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:43.367691 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:43.367994 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 10:52:43.368092 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":40}
I20260927 10:52:43.371600 30372 catalog_manager.cc:1347] Generated new cluster ID: 05ce908a7f0744a3a742c5cb5b5c2b2b
I20260927 10:52:43.371675 30372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:43.409001 30372 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:43.410167 30372 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:43.414798 30372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 348d6e7a0ab145bab1774cf9221b2096: Generated new TSK 0
I20260927 10:52:43.415474 30372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:43.425336 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30284
W20260927 10:52:43.430738 30269 connection.cc:535] client connection to 127.0.0.1:45627 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45627): Connection reset by peer (error 104)
2026-09-27T10:52:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:43 dist-test-slave-p9q6 krb5kdc[30383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:43 dist-test-slave-p9q6 krb5kdc[30383](info): set up 2 sockets
Sep 27 10:52:43 dist-test-slave-p9q6 krb5kdc[30383](info): commencing operation
krb5kdc: starting...
W20260927 10:52:44.471005 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:44 dist-test-slave-p9q6 krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:44.589736 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:44.696352 30396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:44.696614 30396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:44.696684 30396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:44.700690 30396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:44.700745 30396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:44.700764 30396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:44.700780 30396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:44.700795 30396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:44.706146 30396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:44.707507 30396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:44.708491 30396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:44.716662 30404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:44.716681 30396 server_base.cc:1034] running on GCE node
W20260927 10:52:44.716701 30402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:44.716675 30401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:44.717455 30396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:44.718493 30396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:44.719651 30396 hybrid_clock.cc:648] HybridClock initialized: now 1790506364719653 us; error 51 us; skew 500 ppm
Sep 27 10:52:44 dist-test-slave-p9q6 krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506364, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:44.722231 30396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:44.723179 30396 webserver.cc:460] Webserver started at http://127.0.0.1:38867/ using document root <none> and password file <none>
I20260927 10:52:44.723838 30396 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:44.723959 30396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:44.724275 30396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:44.726593 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "7557450c2f9d4f65a27104b6a29e410d"
format_stamp: "Formatted at 2026-09-27 10:52:44 on dist-test-slave-p9q6"
server_key: "5be3f13e147673aab564955d2fd8509a"
server_key_iv: "edcab50b505c4f360645a6cef19e28f6"
server_key_version: "encryptionkey@0"
I20260927 10:52:44.727257 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "7557450c2f9d4f65a27104b6a29e410d"
format_stamp: "Formatted at 2026-09-27 10:52:44 on dist-test-slave-p9q6"
server_key: "5be3f13e147673aab564955d2fd8509a"
server_key_iv: "edcab50b505c4f360645a6cef19e28f6"
server_key_version: "encryptionkey@0"
I20260927 10:52:44.731025 30396 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:52:44.733377 30411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:44.734504 30396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:44.734642 30396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "7557450c2f9d4f65a27104b6a29e410d"
format_stamp: "Formatted at 2026-09-27 10:52:44 on dist-test-slave-p9q6"
server_key: "5be3f13e147673aab564955d2fd8509a"
server_key_iv: "edcab50b505c4f360645a6cef19e28f6"
server_key_version: "encryptionkey@0"
I20260927 10:52:44.734803 30396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:44.749254 30396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:44.751906 30396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:44.752103 30396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:44.758795 30396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38199
I20260927 10:52:44.758828 30463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38199 every 8 connection(s)
I20260927 10:52:44.759716 30396 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:44.762310 30464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:44.765497 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30396
I20260927 10:52:44.765669 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:44.765981 9374 external_mini_cluster.cc:1428] Setting key 71c9db143e5c59809f4ebf7705f27ab0
I20260927 10:52:44.768124 30464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d: Bootstrap starting.
I20260927 10:52:44.770243 30464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:44.771015 30464 log.cc:824] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:44.772936 30464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d: No bootstrap required, opened a new log
I20260927 10:52:44.774978 30467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:44.767346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:51527 (local address 127.0.0.1:38199)
0927 10:52:44.767769 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:44.767785 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:44.767818 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:52:44.768495 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:44.768513 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:44.768849 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:44.769177 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:44.769576 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:44.770600 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:44.771555 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:44.772017 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:44.772184 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:44.773135 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:44.773140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:44.773152 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:44.773520 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:44.773570 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:44.773592 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:44.773731 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:44.773908 (+ 177us) server_negotiation.cc:299] Negotiation successful
0927 10:52:44.774040 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":115,"threads_started":1}
I20260927 10:52:44.776011 30464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7557450c2f9d4f65a27104b6a29e410d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38199 } }
I20260927 10:52:44.776242 30464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:44.776321 30464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7557450c2f9d4f65a27104b6a29e410d, State: Initialized, Role: FOLLOWER
I20260927 10:52:44.776813 30464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7557450c2f9d4f65a27104b6a29e410d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38199 } }
I20260927 10:52:44.776979 30464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:44.777045 30464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:44.777144 30464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:44.777803 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:44.778168 30464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7557450c2f9d4f65a27104b6a29e410d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38199 } }
I20260927 10:52:44.778497 30464 leader_election.cc:302] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7557450c2f9d4f65a27104b6a29e410d; no voters:
I20260927 10:52:44.778873 30464 leader_election.cc:288] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:44.778959 30469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:44.779171 30469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [term 1 LEADER]: Becoming Leader. State: Replica: 7557450c2f9d4f65a27104b6a29e410d, State: Running, Role: LEADER
I20260927 10:52:44.779505 30469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7557450c2f9d4f65a27104b6a29e410d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38199 } }
I20260927 10:52:44.780109 30464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:44.780706 30473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7557450c2f9d4f65a27104b6a29e410d. Latest consensus state: current_term: 1 leader_uuid: "7557450c2f9d4f65a27104b6a29e410d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7557450c2f9d4f65a27104b6a29e410d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38199 } } }
I20260927 10:52:44.780846 30473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:44.781273 30472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7557450c2f9d4f65a27104b6a29e410d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7557450c2f9d4f65a27104b6a29e410d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38199 } } }
I20260927 10:52:44.781376 30472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:44.782413 30480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:44.785209 30467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:44.780018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:34105 (local address 127.0.0.1:38199)
0927 10:52:44.780115 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:44.780120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:44.780553 (+ 433us) server_negotiation.cc:407] Connection header received
0927 10:52:44.781244 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:44.781249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:44.781303 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:44.781355 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:44.781745 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:44.782486 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:44.783567 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:44.783769 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:44.783882 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:44.784740 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:44.784743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:44.784746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:44.784758 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:44.784790 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:44.784801 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:44.784895 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:44.785030 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 10:52:44.785063 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:52:44.785605 30480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:44.791756 30480 catalog_manager.cc:1347] Generated new cluster ID: e973869086fe4a7cb2efa51313ace5e7
I20260927 10:52:44.791818 30480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:44.815541 30480 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:44.816497 30480 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:44.826309 30480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7557450c2f9d4f65a27104b6a29e410d: Generated new TSK 0
I20260927 10:52:44.827176 30480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:44.833233 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30396
W20260927 10:52:44.838447 30381 connection.cc:535] client connection to 127.0.0.1:38199 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38199): Connection reset by peer (error 104)
2026-09-27T10:52:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:44 dist-test-slave-p9q6 krb5kdc[30495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:44 dist-test-slave-p9q6 krb5kdc[30495](info): set up 2 sockets
Sep 27 10:52:44 dist-test-slave-p9q6 krb5kdc[30495](info): commencing operation
krb5kdc: starting...
W20260927 10:52:45.862998 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:45 dist-test-slave-p9q6 krb5kdc[30495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:45.985554 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:46.098246 30508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:46.098640 30508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:46.098755 30508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:46.103021 30508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:46.103119 30508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:46.103163 30508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:46.103225 30508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:46.103269 30508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:46.108711 30508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:46.109884 30508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:46.111003 30508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:46.120200 30516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:46.120200 30514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:46.120222 30513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:46.120644 30508 server_base.cc:1034] running on GCE node
I20260927 10:52:46.121291 30508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:46.122562 30508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:46.123790 30508 hybrid_clock.cc:648] HybridClock initialized: now 1790506366123752 us; error 43 us; skew 500 ppm
Sep 27 10:52:46 dist-test-slave-p9q6 krb5kdc[30495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:46.127017 30508 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:46.128047 30508 webserver.cc:460] Webserver started at http://127.0.0.1:34827/ using document root <none> and password file <none>
I20260927 10:52:46.128697 30508 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:46.128800 30508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:46.129060 30508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:46.131402 30508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "5a0468b1bed14447a5c6b82d5f21caa4"
format_stamp: "Formatted at 2026-09-27 10:52:46 on dist-test-slave-p9q6"
server_key: "88f72b4903a1eb7026f5b26f250ecad9"
server_key_iv: "f3bd1682d75ccd4743ce2cfcc6113836"
server_key_version: "encryptionkey@0"
I20260927 10:52:46.132009 30508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "5a0468b1bed14447a5c6b82d5f21caa4"
format_stamp: "Formatted at 2026-09-27 10:52:46 on dist-test-slave-p9q6"
server_key: "88f72b4903a1eb7026f5b26f250ecad9"
server_key_iv: "f3bd1682d75ccd4743ce2cfcc6113836"
server_key_version: "encryptionkey@0"
I20260927 10:52:46.136037 30508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:52:46.138808 30523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:46.140086 30508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:46.140237 30508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "5a0468b1bed14447a5c6b82d5f21caa4"
format_stamp: "Formatted at 2026-09-27 10:52:46 on dist-test-slave-p9q6"
server_key: "88f72b4903a1eb7026f5b26f250ecad9"
server_key_iv: "f3bd1682d75ccd4743ce2cfcc6113836"
server_key_version: "encryptionkey@0"
I20260927 10:52:46.140363 30508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:46.145416 30508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:46.148494 30508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:46.148702 30508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:46.156208 30508 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33195
I20260927 10:52:46.156227 30575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33195 every 8 connection(s)
I20260927 10:52:46.157240 30508 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:46.160066 30576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:46.162153 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30508
I20260927 10:52:46.162321 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:46.162647 9374 external_mini_cluster.cc:1428] Setting key a2dd0163298bc15a0cdf98450f24e0f3
I20260927 10:52:46.166447 30576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4: Bootstrap starting.
I20260927 10:52:46.168445 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:46.164154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:53437 (local address 127.0.0.1:33195)
0927 10:52:46.164610 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:46.164631 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:46.164677 (+ 46us) server_negotiation.cc:407] Connection header received
0927 10:52:46.165539 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:46.165562 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:46.165922 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:46.166281 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:46.167061 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:46.167067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:46.167083 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:46.167096 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:46.167159 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:46.167187 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:46.167329 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:46.167396 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 10:52:46.167542 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":120,"threads_started":1}
I20260927 10:52:46.169143 30576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:46.169903 30576 log.cc:824] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:46.171271 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:46.172049 30576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4: No bootstrap required, opened a new log
I20260927 10:52:46.174981 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:46.173353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:44173 (local address 127.0.0.1:33195)
0927 10:52:46.173472 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:46.173476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:46.173642 (+ 166us) server_negotiation.cc:407] Connection header received
0927 10:52:46.173745 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:46.173748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:46.173780 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:46.173856 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:46.174539 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:46.174543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:46.174545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:46.174557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:46.174585 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:46.174597 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:46.174684 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:46.174818 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 10:52:46.174859 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 10:52:46.176383 30576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33195 } }
I20260927 10:52:46.176602 30576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:46.176676 30576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0468b1bed14447a5c6b82d5f21caa4, State: Initialized, Role: FOLLOWER
I20260927 10:52:46.177078 30576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33195 } }
I20260927 10:52:46.177227 30576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:46.177294 30576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:46.177402 30576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:46.178310 30576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33195 } }
I20260927 10:52:46.178661 30576 leader_election.cc:302] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0468b1bed14447a5c6b82d5f21caa4; no voters:
I20260927 10:52:46.179020 30576 leader_election.cc:288] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:46.179142 30586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:46.179352 30586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [term 1 LEADER]: Becoming Leader. State: Replica: 5a0468b1bed14447a5c6b82d5f21caa4, State: Running, Role: LEADER
I20260927 10:52:46.179904 30586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33195 } }
I20260927 10:52:46.180240 30576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:46.181559 30588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0468b1bed14447a5c6b82d5f21caa4. Latest consensus state: current_term: 1 leader_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33195 } } }
I20260927 10:52:46.181597 30587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0468b1bed14447a5c6b82d5f21caa4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33195 } } }
I20260927 10:52:46.181727 30587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:46.181674 30588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4 [sys.catalog]: This master's current role is: LEADER
W20260927 10:52:46.184724 30601 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:52:46.184821 30601 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:52:46.184895 30598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:46.185555 30598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:46.191063 30598 catalog_manager.cc:1347] Generated new cluster ID: dd2664f8598a49f8874daa32d8ed7436
I20260927 10:52:46.191128 30598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:46.214860 30598 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:46.215224 30598 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:46.220741 30598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a0468b1bed14447a5c6b82d5f21caa4: Generated new TSK 0
I20260927 10:52:46.221405 30598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:46.229108 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30508
2026-09-27T10:52:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:46 dist-test-slave-p9q6 krb5kdc[30607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:46 dist-test-slave-p9q6 krb5kdc[30607](info): set up 2 sockets
Sep 27 10:52:46 dist-test-slave-p9q6 krb5kdc[30607](info): commencing operation
krb5kdc: starting...
W20260927 10:52:47.299122 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:47 dist-test-slave-p9q6 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:47.423385 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:47.532212 30620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:47.532466 30620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:47.532549 30620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:47.536630 30620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:47.536689 30620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:47.536710 30620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:47.536751 30620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:47.536784 30620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:47.542029 30620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:47.543107 30620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:47.544117 30620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:47.552300 30625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:47.552331 30626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:47.552286 30628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:47.552637 30620 server_base.cc:1034] running on GCE node
I20260927 10:52:47.553203 30620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:47.554291 30620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:47.555457 30620 hybrid_clock.cc:648] HybridClock initialized: now 1790506367555444 us; error 25 us; skew 500 ppm
Sep 27 10:52:47 dist-test-slave-p9q6 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:47.558037 30620 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:47.558959 30620 webserver.cc:460] Webserver started at http://127.0.0.1:38635/ using document root <none> and password file <none>
I20260927 10:52:47.559477 30620 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:47.559561 30620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:47.559798 30620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:47.561718 30620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "8aae2472178340e0802467a4b9496f78"
format_stamp: "Formatted at 2026-09-27 10:52:47 on dist-test-slave-p9q6"
server_key: "45f02b1ffd12ad8ee68ba95a94e3ba6c"
server_key_iv: "8a3b477cb2ce1ff6e1095b3d97dae886"
server_key_version: "encryptionkey@0"
I20260927 10:52:47.562244 30620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "8aae2472178340e0802467a4b9496f78"
format_stamp: "Formatted at 2026-09-27 10:52:47 on dist-test-slave-p9q6"
server_key: "45f02b1ffd12ad8ee68ba95a94e3ba6c"
server_key_iv: "8a3b477cb2ce1ff6e1095b3d97dae886"
server_key_version: "encryptionkey@0"
I20260927 10:52:47.565822 30620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:52:47.568152 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:47.569365 30620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 10:52:47.569499 30620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "8aae2472178340e0802467a4b9496f78"
format_stamp: "Formatted at 2026-09-27 10:52:47 on dist-test-slave-p9q6"
server_key: "45f02b1ffd12ad8ee68ba95a94e3ba6c"
server_key_iv: "8a3b477cb2ce1ff6e1095b3d97dae886"
server_key_version: "encryptionkey@0"
I20260927 10:52:47.569609 30620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:47.584882 30620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:47.587852 30620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:47.588061 30620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:47.595310 30620 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45107
I20260927 10:52:47.595333 30687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45107 every 8 connection(s)
I20260927 10:52:47.596262 30620 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:47.598894 30688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:47.599272 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30620
I20260927 10:52:47.599421 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:47.599709 9374 external_mini_cluster.cc:1428] Setting key 6fda0135d73887a4cca18370bec99046
I20260927 10:52:47.605441 30688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78: Bootstrap starting.
I20260927 10:52:47.607990 30688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78: Neither blocks nor log segments found. Creating new log.
Sep 27 10:52:47 dist-test-slave-p9q6 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 10:52:47.608930 30688 log.cc:824] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:47.611270 30688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78: No bootstrap required, opened a new log
I20260927 10:52:47.614434 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:47.601072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:36571 (local address 127.0.0.1:45107)
0927 10:52:47.601490 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:47.601504 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:47.601538 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:52:47.602215 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:47.602232 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:47.602595 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:47.603002 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:47.603804 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:47.604793 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:47.605665 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:47.606315 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:47.606513 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:47.609371 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:47.609380 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:47.609398 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:47.609434 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:47.611447 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:47.612031 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:47.612036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:47.612041 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:47.612149 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:47.612501 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:47.612503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:47.612504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:47.612891 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:47.613049 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:47.613426 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 10:52:47.613662 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":126,"threads_started":1}
I20260927 10:52:47.615372 30688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aae2472178340e0802467a4b9496f78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260927 10:52:47.615612 30688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:47.615691 30688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aae2472178340e0802467a4b9496f78, State: Initialized, Role: FOLLOWER
I20260927 10:52:47.616117 30688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aae2472178340e0802467a4b9496f78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260927 10:52:47.616263 30688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:47.616333 30688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:47.616456 30688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:47.617468 30688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aae2472178340e0802467a4b9496f78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260927 10:52:47.617568 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:47.617873 30688 leader_election.cc:302] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aae2472178340e0802467a4b9496f78; no voters:
I20260927 10:52:47.618214 30688 leader_election.cc:288] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:47.618330 30693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:47.618599 30693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [term 1 LEADER]: Becoming Leader. State: Replica: 8aae2472178340e0802467a4b9496f78, State: Running, Role: LEADER
I20260927 10:52:47.618974 30693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aae2472178340e0802467a4b9496f78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260927 10:52:47.619566 30688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:47.620141 30693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aae2472178340e0802467a4b9496f78. Latest consensus state: current_term: 1 leader_uuid: "8aae2472178340e0802467a4b9496f78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aae2472178340e0802467a4b9496f78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260927 10:52:47.620275 30693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:47.620568 30695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aae2472178340e0802467a4b9496f78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aae2472178340e0802467a4b9496f78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260927 10:52:47.620661 30695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:47.620782 30705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:47.623997 30705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:47.629487 30705 catalog_manager.cc:1347] Generated new cluster ID: e8b19a164bcc4be79bbe49c0fd81edf0
I20260927 10:52:47.629575 30705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:47.630870 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:47.619948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:44199 (local address 127.0.0.1:45107)
0927 10:52:47.620056 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:47.620060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:47.621121 (+ 1061us) server_negotiation.cc:407] Connection header received
0927 10:52:47.621227 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:47.621233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:47.621292 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:47.621395 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:47.622417 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:47.623217 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:47.624569 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:47.624914 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:47.624992 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:47.626989 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:47.626992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:47.626995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:47.627019 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:47.628829 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:47.629458 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:47.629461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:47.629463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:47.629550 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:47.629946 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:47.629950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:47.629952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:47.630162 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:47.630325 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:47.630589 (+ 264us) server_negotiation.cc:299] Negotiation successful
0927 10:52:47.630675 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 10:52:47.647004 30705 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:47.648186 30705 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:47.657366 30705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8aae2472178340e0802467a4b9496f78: Generated new TSK 0
I20260927 10:52:47.658200 30705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:47.669661 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30620
W20260927 10:52:47.674832 30605 connection.cc:535] client connection to 127.0.0.1:45107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45107): Connection reset by peer (error 104)
2026-09-27T10:52:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:47 dist-test-slave-p9q6 krb5kdc[30719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:47 dist-test-slave-p9q6 krb5kdc[30719](info): set up 2 sockets
Sep 27 10:52:47 dist-test-slave-p9q6 krb5kdc[30719](info): commencing operation
krb5kdc: starting...
W20260927 10:52:48.711149 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:48 dist-test-slave-p9q6 krb5kdc[30719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:48.833479 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:48.940156 30732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:48.940462 30732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:48.940541 30732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:48.944531 30732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:48.944613 30732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:48.944651 30732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:48.944687 30732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:48.944725 30732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:48.949774 30732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:48.950891 30732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:48.951941 30732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:48.960194 30740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:48.960249 30737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:48.960296 30732 server_base.cc:1034] running on GCE node
W20260927 10:52:48.960211 30738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:48.960963 30732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:48.962008 30732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:48.963197 30732 hybrid_clock.cc:648] HybridClock initialized: now 1790506368963180 us; error 27 us; skew 500 ppm
Sep 27 10:52:48 dist-test-slave-p9q6 krb5kdc[30719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506368, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:48.971020 30732 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:48.971979 30732 webserver.cc:460] Webserver started at http://127.0.0.1:44185/ using document root <none> and password file <none>
I20260927 10:52:48.972573 30732 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:48.972671 30732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:48.972913 30732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:48.975052 30732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "ebe3888183b045dc98363fa0d21835bd"
format_stamp: "Formatted at 2026-09-27 10:52:48 on dist-test-slave-p9q6"
server_key: "679819dbfd62aa210fe3da5c49dcbda7"
server_key_iv: "66e7be142408c7b4bea61ea4c758840a"
server_key_version: "encryptionkey@0"
I20260927 10:52:48.975610 30732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "ebe3888183b045dc98363fa0d21835bd"
format_stamp: "Formatted at 2026-09-27 10:52:48 on dist-test-slave-p9q6"
server_key: "679819dbfd62aa210fe3da5c49dcbda7"
server_key_iv: "66e7be142408c7b4bea61ea4c758840a"
server_key_version: "encryptionkey@0"
I20260927 10:52:48.979432 30732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 10:52:48.981860 30747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:48.983107 30732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:48.983249 30732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "ebe3888183b045dc98363fa0d21835bd"
format_stamp: "Formatted at 2026-09-27 10:52:48 on dist-test-slave-p9q6"
server_key: "679819dbfd62aa210fe3da5c49dcbda7"
server_key_iv: "66e7be142408c7b4bea61ea4c758840a"
server_key_version: "encryptionkey@0"
I20260927 10:52:48.983372 30732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:48.996191 30732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:48.999090 30732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:48.999274 30732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:49.006331 30732 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43195
I20260927 10:52:49.006357 30799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43195 every 8 connection(s)
I20260927 10:52:49.007357 30732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:49.009325 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30732
I20260927 10:52:49.009467 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:49.009758 9374 external_mini_cluster.cc:1428] Setting key 4db233f1d748800b25c9f07663f6978d
I20260927 10:52:49.010210 30800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:49.016039 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd: Bootstrap starting.
I20260927 10:52:49.018265 30800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:49.018366 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:49.011144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:57467 (local address 127.0.0.1:43195)
0927 10:52:49.011575 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:49.011589 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:49.011623 (+ 34us) server_negotiation.cc:407] Connection header received
0927 10:52:49.012266 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:49.012281 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:49.012602 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:49.012927 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:49.013321 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:49.014279 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:49.015228 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:49.015688 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:49.015861 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:49.016706 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:49.016711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:49.016723 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:49.017115 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:49.017165 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:49.017187 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:49.017324 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:49.017527 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 10:52:49.017662 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":137,"threads_started":1}
I20260927 10:52:49.019068 30800 log.cc:824] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:49.020908 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd: No bootstrap required, opened a new log
I20260927 10:52:49.021126 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:49.025027 30800 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe3888183b045dc98363fa0d21835bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260927 10:52:49.025261 30800 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:49.025353 30800 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe3888183b045dc98363fa0d21835bd, State: Initialized, Role: FOLLOWER
I20260927 10:52:49.025810 30800 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe3888183b045dc98363fa0d21835bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260927 10:52:49.025956 30800 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:49.026052 30800 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:49.026158 30800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:49.027005 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:49.023019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:51989 (local address 127.0.0.1:43195)
0927 10:52:49.023139 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:49.023144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:49.023273 (+ 129us) server_negotiation.cc:407] Connection header received
0927 10:52:49.023384 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:49.023387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:49.023430 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:49.023512 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:49.023740 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:49.024507 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:49.025304 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:49.025599 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:49.025710 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:49.026530 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:49.026534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:49.026536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:49.026548 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:49.026580 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:49.026590 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:49.026683 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:49.026828 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 10:52:49.026872 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:52:49.027115 30800 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe3888183b045dc98363fa0d21835bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260927 10:52:49.027454 30800 leader_election.cc:302] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebe3888183b045dc98363fa0d21835bd; no voters:
I20260927 10:52:49.027793 30800 leader_election.cc:288] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:49.027866 30810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:49.028064 30810 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [term 1 LEADER]: Becoming Leader. State: Replica: ebe3888183b045dc98363fa0d21835bd, State: Running, Role: LEADER
I20260927 10:52:49.028384 30810 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe3888183b045dc98363fa0d21835bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260927 10:52:49.029002 30800 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:49.030064 30812 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebe3888183b045dc98363fa0d21835bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe3888183b045dc98363fa0d21835bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260927 10:52:49.030189 30812 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:49.030524 30811 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebe3888183b045dc98363fa0d21835bd. Latest consensus state: current_term: 1 leader_uuid: "ebe3888183b045dc98363fa0d21835bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe3888183b045dc98363fa0d21835bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260927 10:52:49.030661 30811 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:49.030987 30819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:49.033663 30819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:49.038537 30819 catalog_manager.cc:1347] Generated new cluster ID: e632d84a02ba42d7a0b93e1e27429400
I20260927 10:52:49.038600 30819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:49.059958 30819 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:49.061187 30819 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:49.076233 30819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebe3888183b045dc98363fa0d21835bd: Generated new TSK 0
I20260927 10:52:49.076967 30819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:49.085600 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30732
W20260927 10:52:49.091097 30717 connection.cc:535] client connection to 127.0.0.1:43195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43195): Connection reset by peer (error 104)
2026-09-27T10:52:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:49 dist-test-slave-p9q6 krb5kdc[30831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:49 dist-test-slave-p9q6 krb5kdc[30831](info): set up 2 sockets
Sep 27 10:52:49 dist-test-slave-p9q6 krb5kdc[30831](info): commencing operation
krb5kdc: starting...
W20260927 10:52:50.114987 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:50 dist-test-slave-p9q6 krb5kdc[30831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 10:52:50.235898 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:50.343987 30844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:50.344259 30844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:50.344298 30844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:50.349972 30844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:50.350054 30844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:50.350082 30844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:50.350104 30844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:50.350163 30844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:50.357235 30844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:50.358285 30844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:50.359318 30844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:50.367880 30852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:50.367905 30850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:50.367944 30849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:50.368402 30844 server_base.cc:1034] running on GCE node
I20260927 10:52:50.368950 30844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:50.370072 30844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:50.371254 30844 hybrid_clock.cc:648] HybridClock initialized: now 1790506370371241 us; error 32 us; skew 500 ppm
Sep 27 10:52:50 dist-test-slave-p9q6 krb5kdc[30831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:50.373951 30844 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 10:52:50.374899 30844 webserver.cc:460] Webserver started at http://127.0.0.1:46717/ using document root <none> and password file <none>
I20260927 10:52:50.375479 30844 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:50.375574 30844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:50.375792 30844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:50.377733 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "621f87ed06924fbf8d0cb93930369829"
format_stamp: "Formatted at 2026-09-27 10:52:50 on dist-test-slave-p9q6"
server_key: "e5faa01c410aa1cd54ee4698d18b6ff8"
server_key_iv: "b3bd13b9921f716c94c8483ad51ebd7a"
server_key_version: "encryptionkey@0"
I20260927 10:52:50.378300 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "621f87ed06924fbf8d0cb93930369829"
format_stamp: "Formatted at 2026-09-27 10:52:50 on dist-test-slave-p9q6"
server_key: "e5faa01c410aa1cd54ee4698d18b6ff8"
server_key_iv: "b3bd13b9921f716c94c8483ad51ebd7a"
server_key_version: "encryptionkey@0"
I20260927 10:52:50.381917 30844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 10:52:50.384496 30859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:50.385787 30844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:50.385926 30844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "621f87ed06924fbf8d0cb93930369829"
format_stamp: "Formatted at 2026-09-27 10:52:50 on dist-test-slave-p9q6"
server_key: "e5faa01c410aa1cd54ee4698d18b6ff8"
server_key_iv: "b3bd13b9921f716c94c8483ad51ebd7a"
server_key_version: "encryptionkey@0"
I20260927 10:52:50.386034 30844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:50.390530 30844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:50.393500 30844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:50.393692 30844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:50.400547 30844 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34245
I20260927 10:52:50.400568 30911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34245 every 8 connection(s)
I20260927 10:52:50.401472 30844 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:50.401686 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30844
I20260927 10:52:50.401842 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:50.402168 9374 external_mini_cluster.cc:1428] Setting key cfd08a366b208be77ec46cb2fba145d2
I20260927 10:52:50.404621 30912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:50.407407 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:50.403563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:40737 (local address 127.0.0.1:34245)
0927 10:52:50.404002 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:50.404016 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:50.404064 (+ 48us) server_negotiation.cc:407] Connection header received
0927 10:52:50.404692 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:50.404743 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:50.405058 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:50.405440 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:50.406213 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:50.406219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:50.406236 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:50.406249 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:50.406314 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:50.406343 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:50.406487 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:50.406555 (+ 68us) server_negotiation.cc:299] Negotiation successful
0927 10:52:50.406727 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":134,"threads_started":1}
I20260927 10:52:50.410259 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:50.410900 30912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829: Bootstrap starting.
I20260927 10:52:50.413395 30912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:50.414047 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:50.412421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:35197 (local address 127.0.0.1:34245)
0927 10:52:50.412537 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:50.412544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:50.412709 (+ 165us) server_negotiation.cc:407] Connection header received
0927 10:52:50.412819 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:50.412821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:50.412854 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:50.412921 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:50.413637 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:50.413640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:50.413642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:50.413653 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:50.413677 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:50.413687 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:50.413756 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:50.413870 (+ 114us) server_negotiation.cc:299] Negotiation successful
0927 10:52:50.413903 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:52:50.414328 30912 log.cc:824] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:50.416260 30912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829: No bootstrap required, opened a new log
I20260927 10:52:50.419735 30912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621f87ed06924fbf8d0cb93930369829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260927 10:52:50.420017 30912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:50.420077 30912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621f87ed06924fbf8d0cb93930369829, State: Initialized, Role: FOLLOWER
I20260927 10:52:50.420475 30912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621f87ed06924fbf8d0cb93930369829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260927 10:52:50.420619 30912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:50.420687 30912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:50.420780 30912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:50.421660 30912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621f87ed06924fbf8d0cb93930369829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260927 10:52:50.421993 30912 leader_election.cc:302] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 621f87ed06924fbf8d0cb93930369829; no voters:
I20260927 10:52:50.422310 30912 leader_election.cc:288] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:50.422386 30922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:50.422580 30922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [term 1 LEADER]: Becoming Leader. State: Replica: 621f87ed06924fbf8d0cb93930369829, State: Running, Role: LEADER
I20260927 10:52:50.422937 30922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621f87ed06924fbf8d0cb93930369829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260927 10:52:50.423607 30912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:50.424561 30924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "621f87ed06924fbf8d0cb93930369829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621f87ed06924fbf8d0cb93930369829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } } }
I20260927 10:52:50.424686 30924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:50.424968 30923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 621f87ed06924fbf8d0cb93930369829. Latest consensus state: current_term: 1 leader_uuid: "621f87ed06924fbf8d0cb93930369829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621f87ed06924fbf8d0cb93930369829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } } }
I20260927 10:52:50.425041 30923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:50.425066 30931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:50.428146 30931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:50.433593 30931 catalog_manager.cc:1347] Generated new cluster ID: f9c7a15f2ce74b059608c7fbbd27be84
I20260927 10:52:50.433662 30931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:50.444217 30931 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:50.444582 30931 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:50.453559 30931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 621f87ed06924fbf8d0cb93930369829: Generated new TSK 0
I20260927 10:52:50.454213 30931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:50.468945 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30844
2026-09-27T10:52:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:50 dist-test-slave-p9q6 krb5kdc[30943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:50 dist-test-slave-p9q6 krb5kdc[30943](info): set up 2 sockets
Sep 27 10:52:50 dist-test-slave-p9q6 krb5kdc[30943](info): commencing operation
krb5kdc: starting...
W20260927 10:52:51.538928 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:51 dist-test-slave-p9q6 krb5kdc[30943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:51.662024 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42407
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:42407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:51.773080 30956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:51.773350 30956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:51.773391 30956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:51.777370 30956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:51.777424 30956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:51.777443 30956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:51.777460 30956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:51.777482 30956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:51.782618 30956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:42407
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:42407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:51.783624 30956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:51.784588 30956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:51.792840 30962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:51.792860 30964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:51.792950 30961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:51.793125 30956 server_base.cc:1034] running on GCE node
I20260927 10:52:51.793830 30956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:51.795127 30956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:51.796329 30956 hybrid_clock.cc:648] HybridClock initialized: now 1790506371796297 us; error 42 us; skew 500 ppm
Sep 27 10:52:51 dist-test-slave-p9q6 krb5kdc[30943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506371, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:51.799263 30956 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:51.800231 30956 webserver.cc:460] Webserver started at http://127.9.39.190:36183/ using document root <none> and password file <none>
I20260927 10:52:51.800880 30956 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:51.800987 30956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:51.801244 30956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:51.803577 30956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "c403980614d642cb80c35b28026591d7"
format_stamp: "Formatted at 2026-09-27 10:52:51 on dist-test-slave-p9q6"
server_key: "5caba21dd725ec21e9bf913e03f9e5d2"
server_key_iv: "5e568f4eb4aedd2fea62e3282369696e"
server_key_version: "encryptionkey@0"
I20260927 10:52:51.804360 30956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "c403980614d642cb80c35b28026591d7"
format_stamp: "Formatted at 2026-09-27 10:52:51 on dist-test-slave-p9q6"
server_key: "5caba21dd725ec21e9bf913e03f9e5d2"
server_key_iv: "5e568f4eb4aedd2fea62e3282369696e"
server_key_version: "encryptionkey@0"
I20260927 10:52:51.810720 30956 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260927 10:52:51.814134 30971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:51.815945 30956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 10:52:51.816110 30956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "c403980614d642cb80c35b28026591d7"
format_stamp: "Formatted at 2026-09-27 10:52:51 on dist-test-slave-p9q6"
server_key: "5caba21dd725ec21e9bf913e03f9e5d2"
server_key_iv: "5e568f4eb4aedd2fea62e3282369696e"
server_key_version: "encryptionkey@0"
I20260927 10:52:51.816231 30956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:51.838028 30956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:51.842180 30956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:51.842399 30956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:51.852414 30956 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:42407
I20260927 10:52:51.852424 31023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:42407 every 8 connection(s)
I20260927 10:52:51.853763 30956 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:51.856630 31024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:51.858047 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 30956
I20260927 10:52:51.858194 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:51.858490 9374 external_mini_cluster.cc:1428] Setting key 76818837fd0fc60bc395bb1429d3cff8
I20260927 10:52:51.863327 31024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7: Bootstrap starting.
I20260927 10:52:51.865845 31024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:51.866561 31024 log.cc:824] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7: Log is configured to *not* fsync() on all Append() calls
Sep 27 10:52:51 dist-test-slave-p9q6 krb5kdc[30943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:51.868551 31024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7: No bootstrap required, opened a new log
I20260927 10:52:51.871878 31024 raft_consensus.cc:348] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c403980614d642cb80c35b28026591d7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42407 } }
I20260927 10:52:51.872193 31024 raft_consensus.cc:374] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:51.872283 31024 raft_consensus.cc:729] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c403980614d642cb80c35b28026591d7, State: Initialized, Role: FOLLOWER
I20260927 10:52:51.872702 31024 consensus_queue.cc:260] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c403980614d642cb80c35b28026591d7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42407 } }
I20260927 10:52:51.872857 31024 raft_consensus.cc:388] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:51.872941 31024 raft_consensus.cc:482] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:51.873029 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:51.859848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56613 (local address 127.9.39.190:42407)
0927 10:52:51.860231 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:51.860249 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:51.860287 (+ 38us) server_negotiation.cc:407] Connection header received
0927 10:52:51.861086 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:51.861109 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:51.861538 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:51.861882 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:51.862535 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:51.863979 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:51.864809 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:51.865473 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:51.865629 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:51.868192 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:51.868198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:51.868213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:51.868244 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:51.869984 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:51.870494 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:51.870500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:51.870504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:51.870607 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:51.870979 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:51.870983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:51.870984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:51.871344 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:51.871581 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:51.871869 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 10:52:51.872085 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":105,"threads_started":1}
I20260927 10:52:51.873036 31024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:51.874042 31024 raft_consensus.cc:504] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c403980614d642cb80c35b28026591d7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42407 } }
I20260927 10:52:51.874379 31024 leader_election.cc:302] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c403980614d642cb80c35b28026591d7; no voters:
I20260927 10:52:51.874686 31024 leader_election.cc:288] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:51.874809 31029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:51.875048 31029 raft_consensus.cc:686] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [term 1 LEADER]: Becoming Leader. State: Replica: c403980614d642cb80c35b28026591d7, State: Running, Role: LEADER
I20260927 10:52:51.875414 31029 consensus_queue.cc:237] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c403980614d642cb80c35b28026591d7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42407 } }
I20260927 10:52:51.876008 31024 sys_catalog.cc:564] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:51.876153 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:51.876988 31030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c403980614d642cb80c35b28026591d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c403980614d642cb80c35b28026591d7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42407 } } }
I20260927 10:52:51.877105 31030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:51.877318 31030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c403980614d642cb80c35b28026591d7. Latest consensus state: current_term: 1 leader_uuid: "c403980614d642cb80c35b28026591d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c403980614d642cb80c35b28026591d7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 42407 } } }
I20260927 10:52:51.877391 31030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:51.878364 31038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:51.881919 31038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:51.887638 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:51.878641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33185 (local address 127.9.39.190:42407)
0927 10:52:51.878769 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:51.878773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:51.879110 (+ 337us) server_negotiation.cc:407] Connection header received
0927 10:52:51.879200 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:51.879202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:51.879238 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:51.879354 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:51.880136 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:51.880879 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:51.881986 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:51.882311 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:51.882387 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:51.884201 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:51.884204 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:51.884207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:51.884230 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:51.885701 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:51.886171 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:51.886174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:51.886176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:51.886283 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:51.886711 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:51.886715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:51.886717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:51.886942 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:51.887093 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:51.887381 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 10:52:51.887480 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 10:52:51.888378 31038 catalog_manager.cc:1347] Generated new cluster ID: a3d05757f21b4558b80a9814dfb7e21e
I20260927 10:52:51.888484 31038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:51.905700 31038 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:51.906646 31038 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:51.911624 31038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c403980614d642cb80c35b28026591d7: Generated new TSK 0
I20260927 10:52:51.912362 31038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:51.921223 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 30956
W20260927 10:52:51.926855 30941 connection.cc:535] client connection to 127.9.39.190:42407 recv error: Network error: failed to read from TLS socket (remote: 127.9.39.190:42407): Connection reset by peer (error 104)
2026-09-27T10:52:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:51 dist-test-slave-p9q6 krb5kdc[31055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:51 dist-test-slave-p9q6 krb5kdc[31055](info): set up 2 sockets
Sep 27 10:52:51 dist-test-slave-p9q6 krb5kdc[31055](info): commencing operation
krb5kdc: starting...
W20260927 10:52:52.959192 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:53 dist-test-slave-p9q6 krb5kdc[31055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:53.086122 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39175
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:53.192981 31068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:53.193305 31068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:53.193382 31068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:53.197444 31068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:53.197539 31068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:53.197579 31068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:53.197640 31068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:53.197680 31068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:53.203028 31068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:53.204159 31068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:53.205327 31068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:53.214342 31076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:53.214439 31073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:53.214532 31074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:53.214591 31068 server_base.cc:1034] running on GCE node
I20260927 10:52:53.215345 31068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:53.216523 31068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:53.217707 31068 hybrid_clock.cc:648] HybridClock initialized: now 1790506373217688 us; error 34 us; skew 500 ppm
Sep 27 10:52:53 dist-test-slave-p9q6 krb5kdc[31055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506373, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:53.220757 31068 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:53.221664 31068 webserver.cc:460] Webserver started at http://127.9.39.190:38537/ using document root <none> and password file <none>
I20260927 10:52:53.222234 31068 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:53.222363 31068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:53.222599 31068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:53.224742 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890"
format_stamp: "Formatted at 2026-09-27 10:52:53 on dist-test-slave-p9q6"
server_key: "22fe782ef68ca748a8e7daa8c83ee303"
server_key_iv: "058c366cd3617ae99744e6e89ec12c4e"
server_key_version: "encryptionkey@0"
I20260927 10:52:53.225346 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890"
format_stamp: "Formatted at 2026-09-27 10:52:53 on dist-test-slave-p9q6"
server_key: "22fe782ef68ca748a8e7daa8c83ee303"
server_key_iv: "058c366cd3617ae99744e6e89ec12c4e"
server_key_version: "encryptionkey@0"
I20260927 10:52:53.229288 31068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 10:52:53.231971 31083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:53.233325 31068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 10:52:53.233479 31068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890"
format_stamp: "Formatted at 2026-09-27 10:52:53 on dist-test-slave-p9q6"
server_key: "22fe782ef68ca748a8e7daa8c83ee303"
server_key_iv: "058c366cd3617ae99744e6e89ec12c4e"
server_key_version: "encryptionkey@0"
I20260927 10:52:53.233603 31068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:53.247927 31068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:53.251431 31068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:53.251645 31068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:53.259157 31068 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39175
I20260927 10:52:53.259173 31135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39175 every 8 connection(s)
I20260927 10:52:53.260133 31068 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:53.262494 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31068
I20260927 10:52:53.262651 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:53.262991 9374 external_mini_cluster.cc:1428] Setting key 08d45204dca68d6282cdf082e214c929
I20260927 10:52:53.263236 31136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:53.270568 31136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890: Bootstrap starting.
I20260927 10:52:53.272851 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:53.264543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40519 (local address 127.9.39.190:39175)
0927 10:52:53.264944 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:53.264968 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:53.265001 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:52:53.265997 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:53.266014 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:53.266359 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:53.266625 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:53.267046 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:53.268012 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:53.269405 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:53.269892 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:53.270043 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:53.271087 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:53.271092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:53.271105 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:53.271511 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:53.271565 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:53.271589 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:53.271695 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:53.271927 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 10:52:53.272093 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":149,"threads_started":1}
I20260927 10:52:53.273180 31136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:53.274070 31136 log.cc:824] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:53.276058 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:53.276374 31136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890: No bootstrap required, opened a new log
I20260927 10:52:53.280155 31136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39175 } }
I20260927 10:52:53.280375 31136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:53.280432 31136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc21e7c0cb941a9a1fdaa57ade0a890, State: Initialized, Role: FOLLOWER
I20260927 10:52:53.280886 31136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39175 } }
I20260927 10:52:53.281037 31136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:53.281109 31136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:53.281222 31136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:53.281925 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:53.278019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50145 (local address 127.9.39.190:39175)
0927 10:52:53.278116 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:53.278120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:53.278340 (+ 220us) server_negotiation.cc:407] Connection header received
0927 10:52:53.278432 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:53.278435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:53.278479 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:53.278559 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:53.278876 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:53.279358 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:53.280192 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:53.280522 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:53.280619 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:53.281483 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:53.281486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:53.281489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:53.281501 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:53.281533 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:53.281544 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:53.281639 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:53.281759 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 10:52:53.281789 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 10:52:53.282526 31136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39175 } }
I20260927 10:52:53.282898 31136 leader_election.cc:302] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bc21e7c0cb941a9a1fdaa57ade0a890; no voters:
I20260927 10:52:53.283217 31136 leader_election.cc:288] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:53.283282 31146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:53.283569 31146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc21e7c0cb941a9a1fdaa57ade0a890, State: Running, Role: LEADER
I20260927 10:52:53.283921 31146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39175 } }
I20260927 10:52:53.284539 31136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:53.285772 31148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bc21e7c0cb941a9a1fdaa57ade0a890. Latest consensus state: current_term: 1 leader_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39175 } } }
I20260927 10:52:53.285785 31147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc21e7c0cb941a9a1fdaa57ade0a890" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39175 } } }
I20260927 10:52:53.285946 31147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:53.285903 31148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:53.286542 31159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:53.289753 31159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:53.295409 31159 catalog_manager.cc:1347] Generated new cluster ID: 72ec6847283845d7b2531462f09c9757
I20260927 10:52:53.295483 31159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:53.315368 31159 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:53.316529 31159 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:53.325994 31159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bc21e7c0cb941a9a1fdaa57ade0a890: Generated new TSK 0
I20260927 10:52:53.326773 31159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:53.338529 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31068
W20260927 10:52:53.344169 31053 connection.cc:535] client connection to 127.9.39.190:39175 recv error: Network error: failed to read from TLS socket (remote: 127.9.39.190:39175): Connection reset by peer (error 104)
2026-09-27T10:52:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:53 dist-test-slave-p9q6 krb5kdc[31167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:53 dist-test-slave-p9q6 krb5kdc[31167](info): set up 2 sockets
Sep 27 10:52:53 dist-test-slave-p9q6 krb5kdc[31167](info): commencing operation
krb5kdc: starting...
W20260927 10:52:54.450937 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:54 dist-test-slave-p9q6 krb5kdc[31167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:54.574100 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:34201
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:34201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:54.685730 31180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:54.686074 31180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:54.686151 31180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:54.690277 31180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:54.690380 31180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:54.690420 31180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:54.690479 31180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:54.690519 31180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:54.695940 31180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:34201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:34201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:54.697069 31180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:54.698136 31180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:54.707959 31185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:54.707959 31188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:54.707965 31186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:54.708076 31180 server_base.cc:1034] running on GCE node
I20260927 10:52:54.708869 31180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:54.710131 31180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:54.711323 31180 hybrid_clock.cc:648] HybridClock initialized: now 1790506374711304 us; error 32 us; skew 500 ppm
Sep 27 10:52:54 dist-test-slave-p9q6 krb5kdc[31167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506374, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:54.719905 31180 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:54.720822 31180 webserver.cc:460] Webserver started at http://127.9.39.190:46487/ using document root <none> and password file <none>
I20260927 10:52:54.721349 31180 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:54.721413 31180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:54.721589 31180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:54.723678 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "51678498f44742f1ba7f358c98efe51a"
format_stamp: "Formatted at 2026-09-27 10:52:54 on dist-test-slave-p9q6"
server_key: "fd7ae668bb2013e09599595d70a0d304"
server_key_iv: "946337245d89d37b315aa9fbe5b3bd26"
server_key_version: "encryptionkey@0"
I20260927 10:52:54.724234 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "51678498f44742f1ba7f358c98efe51a"
format_stamp: "Formatted at 2026-09-27 10:52:54 on dist-test-slave-p9q6"
server_key: "fd7ae668bb2013e09599595d70a0d304"
server_key_iv: "946337245d89d37b315aa9fbe5b3bd26"
server_key_version: "encryptionkey@0"
I20260927 10:52:54.728135 31180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:52:54.730674 31195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:54.732020 31180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:54.732169 31180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "51678498f44742f1ba7f358c98efe51a"
format_stamp: "Formatted at 2026-09-27 10:52:54 on dist-test-slave-p9q6"
server_key: "fd7ae668bb2013e09599595d70a0d304"
server_key_iv: "946337245d89d37b315aa9fbe5b3bd26"
server_key_version: "encryptionkey@0"
I20260927 10:52:54.732280 31180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:54.736805 31180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:54.740031 31180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:54.740218 31180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:54.746897 31180 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:34201
I20260927 10:52:54.747179 31247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:34201 every 8 connection(s)
I20260927 10:52:54.747824 31180 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:54.750349 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31180
I20260927 10:52:54.750492 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:54.750845 31248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:54.750854 9374 external_mini_cluster.cc:1428] Setting key d750cc42910a39cabfb373775a8af92e
I20260927 10:52:54.756461 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:54.752163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45649 (local address 127.9.39.190:34201)
0927 10:52:54.752566 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:54.752585 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:54.752626 (+ 41us) server_negotiation.cc:407] Connection header received
0927 10:52:54.753591 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:54.753615 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:54.753977 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:54.754346 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:54.755099 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:54.755104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:54.755118 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:54.755129 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:54.755192 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:54.755217 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:54.755368 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:54.755443 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 10:52:54.755587 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260927 10:52:54.758072 31248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a: Bootstrap starting.
I20260927 10:52:54.759198 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:54.760461 31248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:54.761236 31248 log.cc:824] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:54.762840 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:54.761231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40619 (local address 127.9.39.190:34201)
0927 10:52:54.761372 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:54.761378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:54.761504 (+ 126us) server_negotiation.cc:407] Connection header received
0927 10:52:54.761616 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:54.761621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:54.761662 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:54.761745 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:54.762479 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:54.762482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:54.762485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:54.762496 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:54.762522 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:54.762534 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:54.762618 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:54.762680 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 10:52:54.762736 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 10:52:54.763646 31248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a: No bootstrap required, opened a new log
I20260927 10:52:54.767373 31248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51678498f44742f1ba7f358c98efe51a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34201 } }
I20260927 10:52:54.767596 31248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:54.767628 31248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51678498f44742f1ba7f358c98efe51a, State: Initialized, Role: FOLLOWER
I20260927 10:52:54.768057 31248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51678498f44742f1ba7f358c98efe51a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34201 } }
I20260927 10:52:54.768177 31248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:54.768222 31248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:54.768293 31248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:54.769117 31248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51678498f44742f1ba7f358c98efe51a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34201 } }
I20260927 10:52:54.769479 31248 leader_election.cc:302] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51678498f44742f1ba7f358c98efe51a; no voters:
I20260927 10:52:54.769784 31248 leader_election.cc:288] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:54.769901 31258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:54.770145 31258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [term 1 LEADER]: Becoming Leader. State: Replica: 51678498f44742f1ba7f358c98efe51a, State: Running, Role: LEADER
I20260927 10:52:54.770519 31258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51678498f44742f1ba7f358c98efe51a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34201 } }
I20260927 10:52:54.771045 31248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:54.772212 31260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51678498f44742f1ba7f358c98efe51a. Latest consensus state: current_term: 1 leader_uuid: "51678498f44742f1ba7f358c98efe51a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51678498f44742f1ba7f358c98efe51a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34201 } } }
I20260927 10:52:54.772212 31259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51678498f44742f1ba7f358c98efe51a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51678498f44742f1ba7f358c98efe51a" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 34201 } } }
I20260927 10:52:54.772413 31259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:54.772413 31260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:54.773240 31268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:54.776131 31268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:54.781401 31268 catalog_manager.cc:1347] Generated new cluster ID: 1a8d347353114673b80e327b439b1713
I20260927 10:52:54.781482 31268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:54.796311 31268 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:54.796680 31268 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:54.806314 31268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51678498f44742f1ba7f358c98efe51a: Generated new TSK 0
I20260927 10:52:54.807031 31268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:54.810444 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31180
2026-09-27T10:52:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:54 dist-test-slave-p9q6 krb5kdc[31279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:54 dist-test-slave-p9q6 krb5kdc[31279](info): set up 2 sockets
Sep 27 10:52:54 dist-test-slave-p9q6 krb5kdc[31279](info): commencing operation
krb5kdc: starting...
W20260927 10:52:55.862970 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:55 dist-test-slave-p9q6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:55.985394 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40027
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:56.095235 31292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:56.095498 31292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:56.095538 31292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:56.100169 31292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:56.100236 31292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:56.100255 31292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:56.100272 31292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:56.100318 31292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:56.105655 31292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:56.106775 31292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:56.107774 31292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:56.116113 31298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:56.116134 31297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:56.116159 31292 server_base.cc:1034] running on GCE node
W20260927 10:52:56.116113 31300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:56.116961 31292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:56.117944 31292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:56.119102 31292 hybrid_clock.cc:648] HybridClock initialized: now 1790506376119080 us; error 28 us; skew 500 ppm
Sep 27 10:52:56 dist-test-slave-p9q6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506376, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:56.121954 31292 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:56.122929 31292 webserver.cc:460] Webserver started at http://127.9.39.190:33133/ using document root <none> and password file <none>
I20260927 10:52:56.123508 31292 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:56.123600 31292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:56.123829 31292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:56.125785 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "c02eb95818e248deaf7e9345c776fd9e"
format_stamp: "Formatted at 2026-09-27 10:52:56 on dist-test-slave-p9q6"
server_key: "2675042573cf2c3fdc396c0b8bcba75d"
server_key_iv: "245ad1d69e0a3a12d5e277968510d73d"
server_key_version: "encryptionkey@0"
I20260927 10:52:56.126358 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "c02eb95818e248deaf7e9345c776fd9e"
format_stamp: "Formatted at 2026-09-27 10:52:56 on dist-test-slave-p9q6"
server_key: "2675042573cf2c3fdc396c0b8bcba75d"
server_key_iv: "245ad1d69e0a3a12d5e277968510d73d"
server_key_version: "encryptionkey@0"
I20260927 10:52:56.130121 31292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 10:52:56.132629 31307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:56.133934 31292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:56.134089 31292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "c02eb95818e248deaf7e9345c776fd9e"
format_stamp: "Formatted at 2026-09-27 10:52:56 on dist-test-slave-p9q6"
server_key: "2675042573cf2c3fdc396c0b8bcba75d"
server_key_iv: "245ad1d69e0a3a12d5e277968510d73d"
server_key_version: "encryptionkey@0"
I20260927 10:52:56.134207 31292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:56.147977 31292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:56.151446 31292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:56.151659 31292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:56.158663 31292 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:40027
I20260927 10:52:56.158751 31359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:40027 every 8 connection(s)
I20260927 10:52:56.159693 31292 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:56.161607 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31292
I20260927 10:52:56.161777 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:56.162066 9374 external_mini_cluster.cc:1428] Setting key 0c5f2e0f59e50615f6134621a1e18d77
I20260927 10:52:56.163046 31360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:56.170207 31360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e: Bootstrap starting.
Sep 27 10:52:56 dist-test-slave-p9q6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:52:56.172861 31360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:56.173836 31360 log.cc:824] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:56.176249 31360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e: No bootstrap required, opened a new log
I20260927 10:52:56.177910 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:56.163643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34267 (local address 127.9.39.190:40027)
0927 10:52:56.164101 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:56.164119 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:56.164162 (+ 43us) server_negotiation.cc:407] Connection header received
0927 10:52:56.165024 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:56.165049 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:56.165526 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:56.165913 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:56.166623 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:56.168158 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:56.169050 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:56.169749 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:56.169911 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:56.172741 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:56.172751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:56.172771 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:56.172810 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:56.174809 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:56.175227 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:56.175233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:56.175239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:56.175391 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:56.175651 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:56.175654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:56.175655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:56.176157 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:56.176370 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:56.176667 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 10:52:56.176964 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":127,"threads_started":1}
I20260927 10:52:56.180644 31360 raft_consensus.cc:348] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c02eb95818e248deaf7e9345c776fd9e" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40027 } }
I20260927 10:52:56.180900 31360 raft_consensus.cc:374] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:56.180958 31360 raft_consensus.cc:729] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c02eb95818e248deaf7e9345c776fd9e, State: Initialized, Role: FOLLOWER
I20260927 10:52:56.181164 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:56.181419 31360 consensus_queue.cc:260] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c02eb95818e248deaf7e9345c776fd9e" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40027 } }
I20260927 10:52:56.181560 31360 raft_consensus.cc:388] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:56.181625 31360 raft_consensus.cc:482] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:56.181723 31360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:56.182760 31360 raft_consensus.cc:504] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c02eb95818e248deaf7e9345c776fd9e" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40027 } }
I20260927 10:52:56.183149 31360 leader_election.cc:302] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c02eb95818e248deaf7e9345c776fd9e; no voters:
I20260927 10:52:56.183557 31360 leader_election.cc:288] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:56.183782 31370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:56.184041 31370 raft_consensus.cc:686] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [term 1 LEADER]: Becoming Leader. State: Replica: c02eb95818e248deaf7e9345c776fd9e, State: Running, Role: LEADER
I20260927 10:52:56.184386 31370 consensus_queue.cc:237] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c02eb95818e248deaf7e9345c776fd9e" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40027 } }
I20260927 10:52:56.184932 31360 sys_catalog.cc:564] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:56.185667 31370 sys_catalog.cc:455] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c02eb95818e248deaf7e9345c776fd9e. Latest consensus state: current_term: 1 leader_uuid: "c02eb95818e248deaf7e9345c776fd9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c02eb95818e248deaf7e9345c776fd9e" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40027 } } }
I20260927 10:52:56.185667 31372 sys_catalog.cc:455] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c02eb95818e248deaf7e9345c776fd9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c02eb95818e248deaf7e9345c776fd9e" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40027 } } }
I20260927 10:52:56.185875 31370 sys_catalog.cc:458] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:56.185875 31372 sys_catalog.cc:458] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:56.186277 31379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:56.190222 31379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:56.192939 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:56.183510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34525 (local address 127.9.39.190:40027)
0927 10:52:56.183622 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:56.183626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:56.183718 (+ 92us) server_negotiation.cc:407] Connection header received
0927 10:52:56.183871 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:56.183874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:56.183920 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:56.184020 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:56.184819 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:56.185637 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:56.186961 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:56.187364 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:56.187506 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:56.189187 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:56.189191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:56.189194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:52:56.189219 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:56.191054 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:56.191546 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:56.191549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:56.191551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:56.191648 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:52:56.191968 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:52:56.191972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:52:56.191973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:52:56.192210 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:56.192353 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:56.192651 (+ 298us) server_negotiation.cc:299] Negotiation successful
0927 10:52:56.192779 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:52:56.196827 31379 catalog_manager.cc:1347] Generated new cluster ID: 5bb746d874974acc9af9b68a4ba1ba8e
I20260927 10:52:56.196908 31379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:56.215547 31379 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:56.216581 31379 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:56.222631 31379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c02eb95818e248deaf7e9345c776fd9e: Generated new TSK 0
I20260927 10:52:56.223445 31379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:56.226876 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31292
2026-09-27T10:52:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:56 dist-test-slave-p9q6 krb5kdc[31391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:56 dist-test-slave-p9q6 krb5kdc[31391](info): set up 2 sockets
Sep 27 10:52:56 dist-test-slave-p9q6 krb5kdc[31391](info): commencing operation
krb5kdc: starting...
W20260927 10:52:57.263089 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:57 dist-test-slave-p9q6 krb5kdc[31391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:57.383322 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39031
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:57.491671 31404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:57.491995 31404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:57.492084 31404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:57.496098 31404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:57.496208 31404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:57.496245 31404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:57.496305 31404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:57.496344 31404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:57.501727 31404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:39031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:57.502893 31404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:57.503976 31404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:57.512571 31412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:57.512722 31410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:57.512638 31404 server_base.cc:1034] running on GCE node
W20260927 10:52:57.512573 31409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:57.513481 31404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:57.514585 31404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:57.515784 31404 hybrid_clock.cc:648] HybridClock initialized: now 1790506377515770 us; error 28 us; skew 500 ppm
Sep 27 10:52:57 dist-test-slave-p9q6 krb5kdc[31391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:57.518524 31404 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:57.519472 31404 webserver.cc:460] Webserver started at http://127.9.39.190:34881/ using document root <none> and password file <none>
I20260927 10:52:57.520020 31404 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:57.520112 31404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:57.520346 31404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:57.522392 31404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "6dac5285c280492cb38e20e3b5c51ddb"
format_stamp: "Formatted at 2026-09-27 10:52:57 on dist-test-slave-p9q6"
server_key: "d5cca29d4ea36877d762b60516adc857"
server_key_iv: "7e54a6929848727afeb4a340c097937f"
server_key_version: "encryptionkey@0"
I20260927 10:52:57.522979 31404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "6dac5285c280492cb38e20e3b5c51ddb"
format_stamp: "Formatted at 2026-09-27 10:52:57 on dist-test-slave-p9q6"
server_key: "d5cca29d4ea36877d762b60516adc857"
server_key_iv: "7e54a6929848727afeb4a340c097937f"
server_key_version: "encryptionkey@0"
I20260927 10:52:57.526856 31404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260927 10:52:57.529270 31419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:57.530499 31404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:52:57.530633 31404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "6dac5285c280492cb38e20e3b5c51ddb"
format_stamp: "Formatted at 2026-09-27 10:52:57 on dist-test-slave-p9q6"
server_key: "d5cca29d4ea36877d762b60516adc857"
server_key_iv: "7e54a6929848727afeb4a340c097937f"
server_key_version: "encryptionkey@0"
I20260927 10:52:57.530804 31404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:57.542541 31404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:57.545650 31404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:57.545864 31404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:57.552788 31404 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:39031
I20260927 10:52:57.552867 31471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:39031 every 8 connection(s)
I20260927 10:52:57.553761 31404 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:57.556452 31472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:57.559355 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31404
I20260927 10:52:57.559499 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:57.559787 9374 external_mini_cluster.cc:1428] Setting key ffe688b76489425dfd489c2f3c87e27d
I20260927 10:52:57.562824 31472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb: Bootstrap starting.
I20260927 10:52:57.565438 31472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:57.566279 31472 log.cc:824] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:57.568616 31472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb: No bootstrap required, opened a new log
I20260927 10:52:57.569820 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:57.561104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38267 (local address 127.9.39.190:39031)
0927 10:52:57.561503 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:57.561523 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:57.561567 (+ 44us) server_negotiation.cc:407] Connection header received
0927 10:52:57.562364 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:57.562385 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:57.562814 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:57.563152 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:57.563571 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:57.565042 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:57.565992 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:57.566736 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:57.566871 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:57.567811 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:57.567817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:57.567833 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:57.568328 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:57.568399 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:57.568431 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:57.568527 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:57.568676 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 10:52:57.568890 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":100,"threads_started":1}
I20260927 10:52:57.572922 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:57.573215 31472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dac5285c280492cb38e20e3b5c51ddb" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39031 } }
I20260927 10:52:57.573545 31472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:57.573621 31472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dac5285c280492cb38e20e3b5c51ddb, State: Initialized, Role: FOLLOWER
I20260927 10:52:57.574147 31472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dac5285c280492cb38e20e3b5c51ddb" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39031 } }
I20260927 10:52:57.574361 31472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:57.574504 31472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:57.574654 31472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:57.575996 31472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dac5285c280492cb38e20e3b5c51ddb" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39031 } }
I20260927 10:52:57.576409 31472 leader_election.cc:302] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dac5285c280492cb38e20e3b5c51ddb; no voters:
I20260927 10:52:57.576754 31472 leader_election.cc:288] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:57.576833 31482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:57.577054 31482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [term 1 LEADER]: Becoming Leader. State: Replica: 6dac5285c280492cb38e20e3b5c51ddb, State: Running, Role: LEADER
I20260927 10:52:57.577409 31482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dac5285c280492cb38e20e3b5c51ddb" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39031 } }
I20260927 10:52:57.578075 31472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:57.578636 31482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dac5285c280492cb38e20e3b5c51ddb. Latest consensus state: current_term: 1 leader_uuid: "6dac5285c280492cb38e20e3b5c51ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dac5285c280492cb38e20e3b5c51ddb" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39031 } } }
I20260927 10:52:57.578832 31482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:57.579290 31489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:57.579403 31483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dac5285c280492cb38e20e3b5c51ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dac5285c280492cb38e20e3b5c51ddb" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 39031 } } }
I20260927 10:52:57.579550 31483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:57.579869 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:57.575260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50095 (local address 127.9.39.190:39031)
0927 10:52:57.575398 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:57.575401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:57.575562 (+ 161us) server_negotiation.cc:407] Connection header received
0927 10:52:57.575672 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:57.575674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:57.575711 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:57.575775 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:57.576152 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:57.576793 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:57.577842 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:52:57.578140 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:52:57.578235 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:52:57.579072 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:57.579076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:57.579080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:57.579093 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:57.579126 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:57.579142 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:57.579221 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:57.579671 (+ 450us) server_negotiation.cc:299] Negotiation successful
0927 10:52:57.579732 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 10:52:57.583039 31489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:57.588680 31489 catalog_manager.cc:1347] Generated new cluster ID: 57f3fee5b21e42949c68b276b915d02e
I20260927 10:52:57.588769 31489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:57.599732 31489 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:57.600734 31489 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:57.605973 31489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dac5285c280492cb38e20e3b5c51ddb: Generated new TSK 0
I20260927 10:52:57.606683 31489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:57.613246 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31404
2026-09-27T10:52:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:57 dist-test-slave-p9q6 krb5kdc[31503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:57 dist-test-slave-p9q6 krb5kdc[31503](info): set up 2 sockets
Sep 27 10:52:57 dist-test-slave-p9q6 krb5kdc[31503](info): commencing operation
krb5kdc: starting...
W20260927 10:52:58.663038 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:52:58 dist-test-slave-p9q6 krb5kdc[31503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:52:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:52:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:52:58.789857 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37305
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:37305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:52:58.900275 31516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:52:58.900604 31516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:52:58.900686 31516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:52:58.904871 31516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:52:58.904982 31516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:52:58.905020 31516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:52:58.905078 31516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:52:58.905117 31516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:52:58.910527 31516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:37305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:52:58.911711 31516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:52:58.912791 31516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:52:58.921528 31522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:58.921522 31521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:52:58.921522 31524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:52:58.921767 31516 server_base.cc:1034] running on GCE node
I20260927 10:52:58.922528 31516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:52:58.923748 31516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:52:58.924921 31516 hybrid_clock.cc:648] HybridClock initialized: now 1790506378924896 us; error 46 us; skew 500 ppm
Sep 27 10:52:58 dist-test-slave-p9q6 krb5kdc[31503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506378, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:52:58.927815 31516 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:52:58.928887 31516 webserver.cc:460] Webserver started at http://127.9.39.190:41931/ using document root <none> and password file <none>
I20260927 10:52:58.929471 31516 fs_manager.cc:359] Metadata directory not provided
I20260927 10:52:58.929562 31516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:52:58.929792 31516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:52:58.931928 31516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "13cd9558b8a94b16b1d4d8364aa20f07"
format_stamp: "Formatted at 2026-09-27 10:52:58 on dist-test-slave-p9q6"
server_key: "64682e2b0d3e25da95eaebc64aed57ae"
server_key_iv: "ff5cfc2e6dfa1570d9edb48850eaaeed"
server_key_version: "encryptionkey@0"
I20260927 10:52:58.932488 31516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "13cd9558b8a94b16b1d4d8364aa20f07"
format_stamp: "Formatted at 2026-09-27 10:52:58 on dist-test-slave-p9q6"
server_key: "64682e2b0d3e25da95eaebc64aed57ae"
server_key_iv: "ff5cfc2e6dfa1570d9edb48850eaaeed"
server_key_version: "encryptionkey@0"
I20260927 10:52:58.936720 31516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 10:52:58.939709 31531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:52:58.941007 31516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:52:58.941171 31516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "13cd9558b8a94b16b1d4d8364aa20f07"
format_stamp: "Formatted at 2026-09-27 10:52:58 on dist-test-slave-p9q6"
server_key: "64682e2b0d3e25da95eaebc64aed57ae"
server_key_iv: "ff5cfc2e6dfa1570d9edb48850eaaeed"
server_key_version: "encryptionkey@0"
I20260927 10:52:58.941320 31516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:52:58.945972 31516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:52:58.949000 31516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:52:58.949198 31516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:52:58.955950 31516 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:37305
I20260927 10:52:58.955960 31583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:37305 every 8 connection(s)
I20260927 10:52:58.956876 31516 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:52:58.959635 31584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:52:58.964869 31584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07: Bootstrap starting.
I20260927 10:52:58.966147 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31516
I20260927 10:52:58.966310 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:52:58.966564 9374 external_mini_cluster.cc:1428] Setting key 4e42040127140ff0bfc0c1ec60c77d84
I20260927 10:52:58.967412 31584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07: Neither blocks nor log segments found. Creating new log.
I20260927 10:52:58.968259 31584 log.cc:824] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07: Log is configured to *not* fsync() on all Append() calls
I20260927 10:52:58.970434 31584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07: No bootstrap required, opened a new log
I20260927 10:52:58.971547 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:58.967652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37219 (local address 127.9.39.190:37305)
0927 10:52:58.968060 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:58.968074 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:58.968107 (+ 33us) server_negotiation.cc:407] Connection header received
0927 10:52:58.968795 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:58.968836 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:58.969107 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:58.969395 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:58.970226 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:58.970231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:58.970241 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:58.970251 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:58.970327 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:58.970347 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:58.970435 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:58.970617 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 10:52:58.970860 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":134,"threads_started":1}
I20260927 10:52:58.974511 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:52:58.974547 31584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37305 } }
I20260927 10:52:58.974860 31584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 10:52:58.974937 31584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13cd9558b8a94b16b1d4d8364aa20f07, State: Initialized, Role: FOLLOWER
I20260927 10:52:58.975334 31584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37305 } }
I20260927 10:52:58.975461 31584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 10:52:58.975545 31584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 10:52:58.975643 31584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:52:58.976716 31584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37305 } }
I20260927 10:52:58.977097 31584 leader_election.cc:302] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13cd9558b8a94b16b1d4d8364aa20f07; no voters:
I20260927 10:52:58.977433 31584 leader_election.cc:288] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 10:52:58.977532 31594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:52:58.977809 31594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [term 1 LEADER]: Becoming Leader. State: Replica: 13cd9558b8a94b16b1d4d8364aa20f07, State: Running, Role: LEADER
I20260927 10:52:58.978180 31594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37305 } }
I20260927 10:52:58.978611 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:52:58.976885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59327 (local address 127.9.39.190:37305)
0927 10:52:58.976986 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 10:52:58.976992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:52:58.977194 (+ 202us) server_negotiation.cc:407] Connection header received
0927 10:52:58.977334 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:52:58.977336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:52:58.977370 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:52:58.977471 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:52:58.978243 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:52:58.978246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:52:58.978249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 10:52:58.978262 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:52:58.978290 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 10:52:58.978307 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:52:58.978389 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 10:52:58.978470 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 10:52:58.978506 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 10:52:58.978935 31584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:52:58.980060 31595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13cd9558b8a94b16b1d4d8364aa20f07. Latest consensus state: current_term: 1 leader_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37305 } } }
I20260927 10:52:58.980194 31595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:58.980532 31596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd9558b8a94b16b1d4d8364aa20f07" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 37305 } } }
I20260927 10:52:58.980608 31603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:52:58.980683 31596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07 [sys.catalog]: This master's current role is: LEADER
I20260927 10:52:58.983695 31603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:52:58.989481 31603 catalog_manager.cc:1347] Generated new cluster ID: 25d78d75bd3140fcb45838b4706e9bef
I20260927 10:52:58.989562 31603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:52:58.997166 31603 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:52:58.997474 31603 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:52:59.012827 31603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13cd9558b8a94b16b1d4d8364aa20f07: Generated new TSK 0
I20260927 10:52:59.013780 31603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 10:52:59.026250 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31516
2026-09-27T10:52:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1413 ms)
[----------] 18 tests from AuthTokenIssuingTest (25395 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:52:59 dist-test-slave-p9q6 krb5kdc[31615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:52:59 dist-test-slave-p9q6 krb5kdc[31615](info): set up 2 sockets
Sep 27 10:52:59 dist-test-slave-p9q6 krb5kdc[31615](info): commencing operation
krb5kdc: starting...
W20260927 10:53:00.070981 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:53:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:53:00.194167 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46369
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46369,127.9.39.189:40321,127.9.39.188:33429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:53:00.303598 31628 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 10:53:00.303969 31628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:53:00.304044 31628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:53:00.304122 31628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:53:00.308152 31628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:53:00.308267 31628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:53:00.308315 31628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:53:00.308352 31628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:53:00.308383 31628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:53:00.313658 31628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46369,127.9.39.189:40321,127.9.39.188:33429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:46369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:53:00.314760 31628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:53:00.315768 31628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:53:00.324388 31634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:00.324549 31628 server_base.cc:1034] running on GCE node
W20260927 10:53:00.324405 31633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:00.324400 31636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:00.325210 31628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:53:00.326315 31628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:53:00.327534 31628 hybrid_clock.cc:648] HybridClock initialized: now 1790506380327504 us; error 48 us; skew 500 ppm
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:53:00.333863 31628 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:53:00.334896 31628 webserver.cc:460] Webserver started at http://127.9.39.190:43311/ using document root <none> and password file <none>
I20260927 10:53:00.335455 31628 fs_manager.cc:359] Metadata directory not provided
I20260927 10:53:00.335546 31628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:53:00.335788 31628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:53:00.337837 31628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "46f0246a4f3d4ab09362c631839bc3fc"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "498dec9b2a678bbe0eefb6d8b1011b7c"
server_key_iv: "226091fd12c9673fc4d83c30d704b614"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.338388 31628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "46f0246a4f3d4ab09362c631839bc3fc"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "498dec9b2a678bbe0eefb6d8b1011b7c"
server_key_iv: "226091fd12c9673fc4d83c30d704b614"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.342374 31628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 10:53:00.344923 31643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:53:00.346238 31628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:53:00.346369 31628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "46f0246a4f3d4ab09362c631839bc3fc"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "498dec9b2a678bbe0eefb6d8b1011b7c"
server_key_iv: "226091fd12c9673fc4d83c30d704b614"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.346498 31628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:53:00.360836 31628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:53:00.363610 31628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:53:00.363766 31628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:53:00.370679 31628 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:46369
I20260927 10:53:00.371029 31695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:46369 every 8 connection(s)
I20260927 10:53:00.371717 31628 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:53:00.374393 31696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:53:00.378075 31696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:00.380575 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31628
I20260927 10:53:00.380710 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:53:00.380995 9374 external_mini_cluster.cc:1428] Setting key 63a7c6b1004da19424c59cf29b2b3156
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.189@KRBTEST.COM" created.
I20260927 10:53:00.393877 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.380347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46369 (local address 127.0.0.1:57698)
0927 10:53:00.380927 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.380969 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.381884 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.382649 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.382654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.383045 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.383320 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.383333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.385078 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.385081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.385841 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.385849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.386847 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.386850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.386927 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.387539 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.387554 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.388995 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.390925 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.390937 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.390949 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.391176 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.391506 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.391511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.391513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.391650 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.392237 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.392275 (+ 38us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.392402 (+ 127us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.392568 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.392878 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":5,"spinlock_wait_cycles":5888,"thread_start_us":149,"threads_started":1}
I20260927 10:53:00.393874 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.381282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.9.39.190:46369)
0927 10:53:00.381559 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.381570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.381612 (+ 42us) server_negotiation.cc:407] Connection header received
0927 10:53:00.382187 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.382191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.382435 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.382627 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.383475 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.384933 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.385966 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.386653 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.386886 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.389138 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.389146 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.389160 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.389192 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.390803 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.391271 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.391276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.391277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.391379 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.391752 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.391755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.391757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.392123 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.392266 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.392632 (+ 366us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.392929 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":180,"spinlock_wait_cycles":3968,"thread_start_us":97,"threads_started":1}
I20260927 10:53:00.396431 31696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:00.397169 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.396923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51784)
0927 10:53:00.397037 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.397110 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 10:53:00.397452 31645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
I20260927 10:53:00.398276 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.398063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51788)
0927 10:53:00.398166 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.398207 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 10:53:00.398516 31696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:40321: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab.
Entry for principal kudu/127.9.39.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab.
I20260927 10:53:00.409373 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:40321
--webserver_interface=127.9.39.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab
--principal=kudu/127.9.39.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46369,127.9.39.189:40321,127.9.39.188:33429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:53:00.447786 31696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } attempt: 1
I20260927 10:53:00.448726 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.448435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51798)
0927 10:53:00.448584 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.448648 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 10:53:00.449524 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.449309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51812)
0927 10:53:00.449419 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.449470 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 10:53:00.449666 31696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:40321: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
I20260927 10:53:00.517831 31696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } attempt: 2
I20260927 10:53:00.518723 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.518405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51820)
0927 10:53:00.518556 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.518620 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260927 10:53:00.519449 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.519240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51834)
0927 10:53:00.519347 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.519399 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 10:53:00.519574 31696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:40321: Network error: Client connection negotiation failed: client connection to 127.9.39.189:40321: connect: Connection refused (error 111)
W20260927 10:53:00.522109 31703 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 10:53:00.522444 31703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:53:00.522505 31703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:53:00.522588 31703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:53:00.526770 31703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:53:00.526857 31703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:53:00.526903 31703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:53:00.526939 31703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:53:00.526973 31703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:53:00.532511 31703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46369,127.9.39.189:40321,127.9.39.188:33429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab
--principal=kudu/127.9.39.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.39.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:53:00.533566 31703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:53:00.534564 31703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:53:00.542896 31709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:00.542925 31708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:00.542903 31711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:00.543321 31703 server_base.cc:1034] running on GCE node
I20260927 10:53:00.543869 31703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:53:00.544914 31703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:53:00.546123 31703 hybrid_clock.cc:648] HybridClock initialized: now 1790506380546084 us; error 41 us; skew 500 ppm
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:53:00.548815 31703 init.cc:377] Logged in from keytab as kudu/127.9.39.189@KRBTEST.COM (short username kudu)
I20260927 10:53:00.549687 31703 webserver.cc:460] Webserver started at http://127.9.39.189:34717/ using document root <none> and password file <none>
I20260927 10:53:00.550256 31703 fs_manager.cc:359] Metadata directory not provided
I20260927 10:53:00.550344 31703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:53:00.550566 31703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:53:00.552840 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data/instance:
uuid: "79bc6a3100a341169ac8bba522cfbb04"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "062e44bfc44d53a38653c8dea9e81a1d"
server_key_iv: "edbb5a2b3fbd598f40f91e729b5662a9"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.553385 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal/instance:
uuid: "79bc6a3100a341169ac8bba522cfbb04"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "062e44bfc44d53a38653c8dea9e81a1d"
server_key_iv: "edbb5a2b3fbd598f40f91e729b5662a9"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.557047 31703 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 10:53:00.559347 31718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:53:00.560506 31703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 10:53:00.560642 31703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal
uuid: "79bc6a3100a341169ac8bba522cfbb04"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "062e44bfc44d53a38653c8dea9e81a1d"
server_key_iv: "edbb5a2b3fbd598f40f91e729b5662a9"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.560756 31703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:53:00.573335 31703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:53:00.576480 31703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:53:00.576695 31703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:53:00.583956 31703 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.189:40321
I20260927 10:53:00.584334 31770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.189:40321 every 8 connection(s)
I20260927 10:53:00.585244 31703 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
I20260927 10:53:00.591418 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31703
I20260927 10:53:00.591584 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-1/wal/instance
I20260927 10:53:00.592005 9374 external_mini_cluster.cc:1428] Setting key 2c046e95ee677989ac79e2f483c23037
I20260927 10:53:00.593715 31771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:53:00.598294 31771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.39.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.188@KRBTEST.COM" created.
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:53:00.610759 31696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } attempt: 3
I20260927 10:53:00.613976 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.600345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.9.39.190:46369)
0927 10:53:00.600488 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.600493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.601525 (+ 1032us) server_negotiation.cc:407] Connection header received
0927 10:53:00.602508 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.602512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.602572 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.602676 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.604256 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.605031 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.606209 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.606663 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.606832 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.609574 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.609579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.609585 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.609622 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.611600 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.612125 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.612129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.612130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.612202 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.612621 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.612623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.612624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.612927 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.613038 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.613635 (+ 597us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.613751 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 10:53:00.615168 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.600835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46369 (local address 127.0.0.1:57708)
0927 10:53:00.601351 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.601410 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.602283 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.602888 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.602900 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.603329 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.604044 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.604067 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.605178 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.605181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.606064 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.606074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.606846 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.606849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.606987 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.607662 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.607679 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.609387 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.611700 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.611705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.611722 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.612001 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.612310 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.612312 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.612314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.612513 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.613038 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.613046 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.613359 (+ 313us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.613571 (+ 212us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.614073 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":131,"thread_start_us":124,"threads_started":1}
I20260927 10:53:00.617165 31771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab.
Entry for principal kudu/127.9.39.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab.
I20260927 10:53:00.621771 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.611243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51850)
0927 10:53:00.611488 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.611505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.611586 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.612681 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.612684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.612941 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.613038 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.613043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.614258 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.614262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.615101 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.615112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.615875 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.615879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.615994 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.616508 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.616522 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.618090 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.620034 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.620038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.620040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.620233 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.620574 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.620578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.620579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.620673 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.621259 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.621263 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.621371 (+ 108us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.621456 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.621602 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260927 10:53:00.621750 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:33429
--webserver_interface=127.9.39.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab
--principal=kudu/127.9.39.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:46369,127.9.39.189:40321,127.9.39.188:33429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:53:00.622664 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.611351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.9.39.189:40321)
0927 10:53:00.612222 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.612233 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.612277 (+ 44us) server_negotiation.cc:407] Connection header received
0927 10:53:00.612317 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.612323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.612559 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.612701 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.613156 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.614091 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.615226 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.615747 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.615875 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.618206 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.618213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.618223 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.618243 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.619921 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.620361 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.620367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.620369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.620459 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.620807 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.620810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.620811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.621124 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.622087 (+ 963us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.622204 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.622383 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":140,"threads_started":1}
I20260927 10:53:00.624496 31696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:00.625170 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.624948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57348)
0927 10:53:00.625057 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.625106 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 10:53:00.625818 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.625582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57358)
0927 10:53:00.625712 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.625759 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260927 10:53:00.626163 31696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:33429: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
I20260927 10:53:00.626946 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.617683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51856)
0927 10:53:00.617780 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.617805 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.617886 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.618268 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.618271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.618433 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.618523 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.618528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.619462 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.619467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.620074 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.620083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.620632 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.620635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.620727 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.621178 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.621190 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.622467 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.625377 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.625383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.625388 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.625626 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.625930 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.625934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.625936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.626026 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.626492 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.626495 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.626595 (+ 100us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.626681 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.626845 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 10:53:00.627172 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.617749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.9.39.189:40321)
0927 10:53:00.618068 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.618077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.618091 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:53:00.618124 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.618130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.618180 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.618260 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.618652 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.619336 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.620217 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.620507 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.620629 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.623183 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.623191 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.623195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.623226 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.625268 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.625729 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.625735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.625739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.625826 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.626125 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.626131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.626136 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.626372 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.626479 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.626794 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.626937 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":250,"thread_start_us":117,"threads_started":1}
I20260927 10:53:00.627722 31771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:00.628347 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.628116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57360)
0927 10:53:00.628223 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.628299 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 10:53:00.628546 31722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
I20260927 10:53:00.629315 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.629117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57368)
0927 10:53:00.629217 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.629260 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 10:53:00.629468 31771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:33429: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
I20260927 10:53:00.657317 31696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } attempt: 1
I20260927 10:53:00.658175 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.657888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57380)
0927 10:53:00.658034 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.658097 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 10:53:00.658910 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.658675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57390)
0927 10:53:00.658811 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.658859 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260927 10:53:00.659039 31696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:33429: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
I20260927 10:53:00.678761 31771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } attempt: 1
I20260927 10:53:00.679649 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.679304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57402)
0927 10:53:00.679484 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.679558 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260927 10:53:00.680388 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.680170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57418)
0927 10:53:00.680279 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.680330 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 10:53:00.680564 31771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:33429: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
I20260927 10:53:00.734210 31696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } attempt: 2
I20260927 10:53:00.735033 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.734755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57432)
0927 10:53:00.734887 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.734957 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 10:53:00.735754 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.735551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57442)
0927 10:53:00.735647 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.735706 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 10:53:00.735868 31696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:33429: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
W20260927 10:53:00.736186 31778 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 10:53:00.736498 31778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:53:00.736564 31778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:53:00.736639 31778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:53:00.740729 31778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:53:00.740814 31778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:53:00.740851 31778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:53:00.740909 31778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:53:00.740947 31778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:53:00.746284 31778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:46369,127.9.39.189:40321,127.9.39.188:33429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab
--principal=kudu/127.9.39.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:33429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.39.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:53:00.747489 31778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:53:00.748541 31778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 10:53:00.748798 31771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } attempt: 2
I20260927 10:53:00.749627 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.749321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57454)
0927 10:53:00.749476 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.749543 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 10:53:00.750423 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.750195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57470)
0927 10:53:00.750324 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.750369 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260927 10:53:00.750568 31771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:33429: Network error: Client connection negotiation failed: client connection to 127.9.39.188:33429: connect: Connection refused (error 111)
W20260927 10:53:00.757714 31785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:00.757750 31784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:00.757716 31787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:00.758092 31778 server_base.cc:1034] running on GCE node
I20260927 10:53:00.758800 31778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:53:00.759946 31778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:53:00.761132 31778 hybrid_clock.cc:648] HybridClock initialized: now 1790506380761104 us; error 51 us; skew 500 ppm
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:53:00.763972 31778 init.cc:377] Logged in from keytab as kudu/127.9.39.188@KRBTEST.COM (short username kudu)
I20260927 10:53:00.764849 31778 webserver.cc:460] Webserver started at http://127.9.39.188:33227/ using document root <none> and password file <none>
I20260927 10:53:00.765444 31778 fs_manager.cc:359] Metadata directory not provided
I20260927 10:53:00.765535 31778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:53:00.765761 31778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:53:00.767849 31778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data/instance:
uuid: "e735698e1ddf4a73b4fab37d343cdb92"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "f1923a8c397c93515ff472d3cf8b17f0"
server_key_iv: "005c6e70ff310d6d6b3221ea0def6a72"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.768448 31778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal/instance:
uuid: "e735698e1ddf4a73b4fab37d343cdb92"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "f1923a8c397c93515ff472d3cf8b17f0"
server_key_iv: "005c6e70ff310d6d6b3221ea0def6a72"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.772156 31778 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 10:53:00.774756 31794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:53:00.776006 31778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:53:00.776160 31778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal
uuid: "e735698e1ddf4a73b4fab37d343cdb92"
format_stamp: "Formatted at 2026-09-27 10:53:00 on dist-test-slave-p9q6"
server_key: "f1923a8c397c93515ff472d3cf8b17f0"
server_key_iv: "005c6e70ff310d6d6b3221ea0def6a72"
server_key_version: "encryptionkey@0"
I20260927 10:53:00.776320 31778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:53:00.789371 31778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:53:00.792905 31778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:53:00.793112 31778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:53:00.800191 31778 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.188:33429
I20260927 10:53:00.800475 31846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.188:33429 every 8 connection(s)
I20260927 10:53:00.801250 31778 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
I20260927 10:53:00.804214 31847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:53:00.807984 31847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:00.809576 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 31778
I20260927 10:53:00.809823 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790506104903010-9374-0/minicluster-data/master-2/wal/instance
I20260927 10:53:00.810132 9374 external_mini_cluster.cc:1428] Setting key dbb810a61356b97b75de58f9e5a13dda
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
I20260927 10:53:00.821941 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.809991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.9.39.190:46369)
0927 10:53:00.810158 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.810163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.811133 (+ 970us) server_negotiation.cc:407] Connection header received
0927 10:53:00.812265 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.812268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.812337 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.812428 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.813268 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.813767 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.814774 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.814966 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.815068 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.817834 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.817839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.817842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.817870 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.819555 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.820187 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.820191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.820192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.820294 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.820667 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.820669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.820670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.820988 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.821093 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.821613 (+ 520us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.821733 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 10:53:00.822921 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.810895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57482 (local address 127.9.39.188:33429)
0927 10:53:00.811195 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.811207 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.811462 (+ 255us) server_negotiation.cc:407] Connection header received
0927 10:53:00.812083 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.812113 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.812312 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.812534 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.813381 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.814199 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.815063 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.815438 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.815542 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.818330 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.818335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.818345 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.818365 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.820043 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.820546 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.820552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.820553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.820631 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.820927 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.820930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.820932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.821378 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.821595 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.821781 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.821954 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":213,"spinlock_wait_cycles":3456,"thread_start_us":138,"threads_started":1}
I20260927 10:53:00.822924 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.810361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:46369 (local address 127.0.0.1:57724)
0927 10:53:00.810967 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.811028 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.812057 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.812475 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.812482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.812830 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.813129 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.813141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.813870 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.813877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.814592 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.814600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.815110 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.815114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.815365 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.815972 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.815995 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.817655 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.819665 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.819673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.819685 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.820093 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.820388 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.820390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.820391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.820571 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.821092 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.821101 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.821376 (+ 275us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.821552 (+ 176us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.821892 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":79,"spinlock_wait_cycles":1920,"thread_start_us":160,"threads_started":1}
I20260927 10:53:00.824788 31847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:53:00.834555 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.825588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.9.39.189:40321)
0927 10:53:00.825711 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.825726 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.825741 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:53:00.825784 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.825787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.825838 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.825931 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.826392 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.827037 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.827792 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.828163 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.828265 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.830779 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.830783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.830786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.830819 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.832798 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.833253 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.833258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.833260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.833357 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.833661 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.833666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.833668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.833882 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.833994 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.834291 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.834411 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 10:53:00.834547 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.825303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:40321 (local address 127.0.0.1:51858)
0927 10:53:00.825409 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.825424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.825494 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.825982 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.825985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.826147 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.826260 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.826266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.827135 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.827137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.827689 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.827697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.828358 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.828361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.828438 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.828961 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.828972 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.830580 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.832907 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.832909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.832911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.833156 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.833480 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.833482 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.833484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.833563 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.833999 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.834004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.834115 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.834224 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.834365 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 10:53:00.835073 31696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } attempt: 3
I20260927 10:53:00.835709 31847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:00.835826 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.825268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.9.39.190:46369)
0927 10:53:00.825379 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.825383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.825396 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:53:00.825435 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.825440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.825489 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.825560 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.827148 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.827611 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.828657 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.828893 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.828995 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.831891 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.831898 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.831901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.831930 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.833625 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.834065 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.834068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.834070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.834155 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.834438 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.834442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.834444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.834643 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.834926 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.835520 (+ 594us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.835635 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
I20260927 10:53:00.841722 31771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } attempt: 3
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:53:00.845194 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.835535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57498)
0927 10:53:00.835639 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.835652 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.835721 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.836012 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.836017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.836239 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.836354 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.836360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.837522 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.837527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.838424 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.838435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.839052 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.839055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.839154 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.839664 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.839677 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.841265 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.843594 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.843597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.843599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.843860 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.844174 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.844177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.844179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.844266 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.844729 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.844733 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.844822 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.844902 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.845023 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 10:53:00.846057 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.835593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57498 (local address 127.9.39.188:33429)
0927 10:53:00.835703 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.835707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.835723 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:53:00.835814 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.835818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.835875 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.836505 (+ 630us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.836598 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.837389 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.838577 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.838934 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.839041 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.841414 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.841418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.841423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.841452 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.843468 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.843999 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.844002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.844004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.844080 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.844369 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.844371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.844373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.844615 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.845721 (+ 1106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.845791 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.845905 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 10:53:00.846346 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.836196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57500)
0927 10:53:00.836664 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.836679 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.836765 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.837065 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.837067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.837233 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.837307 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.837313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.838772 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.838776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.839491 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.839504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.840024 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.840027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.840106 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.840578 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.840596 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.841975 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.844673 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.844677 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.844680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.844926 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.845248 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.845253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.845256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.845345 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.845808 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.845814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.845912 (+ 98us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.846024 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.846153 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400}
I20260927 10:53:00.847262 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.837161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.9.39.189:40321)
0927 10:53:00.837267 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.837271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.837328 (+ 57us) server_negotiation.cc:407] Connection header received
0927 10:53:00.837440 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.837443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.837479 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.837569 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.838154 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.838618 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.839588 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.839904 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.840017 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.842944 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.842950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.842957 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.842992 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.844910 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.845422 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.845429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.845431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.845518 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.845878 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.845882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.845884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.846098 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.846569 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.847005 (+ 436us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.847118 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 10:53:00 dist-test-slave-p9q6 krb5kdc[31615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
I20260927 10:53:00.849895 31696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: Bootstrap starting.
I20260927 10:53:00.850859 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.836243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.9.39.188:33429)
0927 10:53:00.836597 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.836607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.836751 (+ 144us) server_negotiation.cc:407] Connection header received
0927 10:53:00.836835 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.836838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.836881 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.836973 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.837430 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.838623 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.839624 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.839873 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.840022 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.842147 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.842151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.842154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.842192 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.844354 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.845028 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.845031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.845033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.845117 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.845447 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.845450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.845452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.845671 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.845768 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.846473 (+ 705us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.846598 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":287,"thread_start_us":115,"threads_started":1}
I20260927 10:53:00.852741 31696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: Neither blocks nor log segments found. Creating new log.
I20260927 10:53:00.853667 31696 log.cc:824] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: Log is configured to *not* fsync() on all Append() calls
I20260927 10:53:00.854703 31847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: Bootstrap starting.
I20260927 10:53:00.855507 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.842076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:33429 (local address 127.0.0.1:57502)
0927 10:53:00.842342 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:00.842357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:00.842428 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:00.843178 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:00.843196 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:00.843375 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:00.843488 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.843497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.844579 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.844583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.845381 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:00.845392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.845986 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.845989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:00.846071 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.846825 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:00.846838 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:00.848542 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:00.853620 (+ 5078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.853623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.853625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.853832 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.854396 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:00.854399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:00.854401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:00.854504 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:00.855030 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:00.855034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:00.855130 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 10:53:00.855211 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 10:53:00.855364 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260927 10:53:00.856042 31696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: No bootstrap required, opened a new log
I20260927 10:53:00.856065 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.842567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.9.39.188:33429)
0927 10:53:00.842929 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.842938 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.842953 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:53:00.842989 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.842993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.843037 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.843593 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.843703 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.844466 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.845520 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.845857 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.846300 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.848706 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.848709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.848712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.848743 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.853508 (+ 4765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.854152 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.854157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.854159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.854256 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.854628 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.854631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.854633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.854909 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.855676 (+ 767us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.855769 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.855894 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":155,"threads_started":1}
I20260927 10:53:00.857194 31847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: Neither blocks nor log segments found. Creating new log.
I20260927 10:53:00.857889 31847 log.cc:824] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: Log is configured to *not* fsync() on all Append() calls
I20260927 10:53:00.858873 31771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: Bootstrap starting.
I20260927 10:53:00.859390 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:53:00.859807 31847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: No bootstrap required, opened a new log
I20260927 10:53:00.860764 31670 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "46f0246a4f3d4ab09362c631839bc3fc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57740
I20260927 10:53:00.861338 31771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: Neither blocks nor log segments found. Creating new log.
I20260927 10:53:00.862147 31771 log.cc:824] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: Log is configured to *not* fsync() on all Append() calls
I20260927 10:53:00.862490 31696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
I20260927 10:53:00.862756 31696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f0246a4f3d4ab09362c631839bc3fc, State: Initialized, Role: FOLLOWER
I20260927 10:53:00.863035 31696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
I20260927 10:53:00.863462 31857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.863590 31857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.864312 31696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:53:00.864427 31847 raft_consensus.cc:348] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
I20260927 10:53:00.864642 31847 raft_consensus.cc:729] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e735698e1ddf4a73b4fab37d343cdb92, State: Initialized, Role: FOLLOWER
I20260927 10:53:00.864899 31771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: No bootstrap required, opened a new log
I20260927 10:53:00.864928 31847 consensus_queue.cc:260] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
W20260927 10:53:00.865203 31649 tablet.cc:2369] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 10:53:00.866130 31847 sys_catalog.cc:564] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:53:00.866999 31670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 10:53:00.867127 31670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:53:00.868531 31670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
I20260927 10:53:00.869388 31670 leader_election.cc:288] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [CANDIDATE]: Term 1 election: Requested vote from peers 79bc6a3100a341169ac8bba522cfbb04 (127.9.39.189:40321), e735698e1ddf4a73b4fab37d343cdb92 (127.9.39.188:33429)
W20260927 10:53:00.870079 31869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:53:00.870151 31869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:53:00.870566 31821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46f0246a4f3d4ab09362c631839bc3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e735698e1ddf4a73b4fab37d343cdb92"
I20260927 10:53:00.870791 31821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:53:00.869700 31771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
W20260927 10:53:00.870995 31881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:53:00.871066 31881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:53:00.871781 31771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bc6a3100a341169ac8bba522cfbb04, State: Initialized, Role: FOLLOWER
I20260927 10:53:00.872103 31821 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46f0246a4f3d4ab09362c631839bc3fc in term 1.
I20260927 10:53:00.872156 31771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
I20260927 10:53:00.872632 31861 sys_catalog.cc:455] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.872767 31861 sys_catalog.cc:458] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.872861 31883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.873023 31883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.873670 31771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:53:00.873941 31745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46f0246a4f3d4ab09362c631839bc3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "79bc6a3100a341169ac8bba522cfbb04"
I20260927 10:53:00.874171 31745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:53:00.875232 31647 leader_election.cc:302] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46f0246a4f3d4ab09362c631839bc3fc, e735698e1ddf4a73b4fab37d343cdb92; no voters:
I20260927 10:53:00.875530 31857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:53:00.875625 31745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46f0246a4f3d4ab09362c631839bc3fc in term 1.
I20260927 10:53:00.875795 31857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [term 1 LEADER]: Becoming Leader. State: Replica: 46f0246a4f3d4ab09362c631839bc3fc, State: Running, Role: LEADER
I20260927 10:53:00.876128 31857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } }
I20260927 10:53:00.877938 31899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f0246a4f3d4ab09362c631839bc3fc. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.878047 31899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: This master's current role is: LEADER
I20260927 10:53:00.879016 31903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260927 10:53:00.879208 31900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:53:00.879328 31900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:53:00.881006 31903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:53:00.881667 31778 master_runner.cc:263] No leader master found from master e735698e1ddf4a73b4fab37d343cdb92
I20260927 10:53:00.881729 31778 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:53:00.885968 31628 master_runner.cc:263] No leader master found from master 46f0246a4f3d4ab09362c631839bc3fc
I20260927 10:53:00.886035 31628 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:53:00.892266 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.874125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.9.39.189:40321)
0927 10:53:00.875936 (+ 1811us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.875941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.882489 (+ 6548us) server_negotiation.cc:407] Connection header received
0927 10:53:00.882693 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.882696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.882773 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.882826 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.883884 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.884702 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.885933 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.886276 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.886354 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.888440 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.888444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.888447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.888478 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.890310 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.890854 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.890858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.890860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.890947 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.891272 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.891276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.891278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.891491 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.891620 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.891966 (+ 346us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.892085 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1741}
I20260927 10:53:00.900151 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.881550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57504 (local address 127.9.39.188:33429)
0927 10:53:00.881668 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.881674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.886312 (+ 4638us) server_negotiation.cc:407] Connection header received
0927 10:53:00.886487 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.886490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.886533 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.886582 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.887628 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.888443 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.889599 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.889960 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.890034 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.895914 (+ 5880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.895919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.895922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.895954 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.897950 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.898751 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.898755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.898757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.898854 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.899160 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.899163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.899165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.899406 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.899517 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.899858 (+ 341us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.899984 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 10:53:00.901973 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:00.882619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.9.39.190:46369)
0927 10:53:00.882742 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.882747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.893128 (+ 10381us) server_negotiation.cc:407] Connection header received
0927 10:53:00.893333 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.893336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.893388 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.893440 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.894210 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.894979 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.896066 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:00.896400 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:00.896478 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:00.898077 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:00.898081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:00.898084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:00.898113 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:00.899860 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.900599 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.900603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.900605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.900694 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:00.901016 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:00.901019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:00.901021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:00.901238 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:00.901340 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:00.901694 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 10:53:00.901812 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:53:00.902230 31745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [term 1 FOLLOWER]: Refusing update from remote peer 46f0246a4f3d4ab09362c631839bc3fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:53:00.902230 31821 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [term 1 FOLLOWER]: Refusing update from remote peer 46f0246a4f3d4ab09362c631839bc3fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:53:00.902964 31857 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:53:00.903683 31857 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:53:00.907882 31861 sys_catalog.cc:455] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f0246a4f3d4ab09362c631839bc3fc. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.908012 31861 sys_catalog.cc:458] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.908367 31883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f0246a4f3d4ab09362c631839bc3fc. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.908511 31883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.912393 31857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.912523 31857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: This master's current role is: LEADER
I20260927 10:53:00.913493 31883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.913618 31883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.914306 31899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.914425 31899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc [sys.catalog]: This master's current role is: LEADER
I20260927 10:53:00.914520 31905 mvcc.cc:205] Tried to move back new op lower bound from 7333914136137375744 to 7333914136071778304. Current Snapshot: MvccSnapshot[applied={T|T < 7333914136137375744}]
I20260927 10:53:00.915282 31861 sys_catalog.cc:455] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46f0246a4f3d4ab09362c631839bc3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46f0246a4f3d4ab09362c631839bc3fc" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 46369 } } peers { permanent_uuid: "79bc6a3100a341169ac8bba522cfbb04" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 40321 } } peers { permanent_uuid: "e735698e1ddf4a73b4fab37d343cdb92" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 33429 } } }
I20260927 10:53:00.915380 31861 sys_catalog.cc:458] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:00.915904 31907 mvcc.cc:205] Tried to move back new op lower bound from 7333914136137375744 to 7333914136071778304. Current Snapshot: MvccSnapshot[applied={T|T < 7333914136137375744}]
I20260927 10:53:00.918783 31908 mvcc.cc:205] Tried to move back new op lower bound from 7333914136137375744 to 7333914136071778304. Current Snapshot: MvccSnapshot[applied={T|T < 7333914136137375744}]
I20260927 10:53:00.919725 31903 catalog_manager.cc:1347] Generated new cluster ID: f39803867e8c430dadce43a599a633c6
I20260927 10:53:00.919795 31903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:53:00.951300 31903 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:53:00.952768 31903 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:53:00.963457 31903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46f0246a4f3d4ab09362c631839bc3fc: Generated new TSK 0
I20260927 10:53:00.964440 31903 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 10:53:00.989674 31853 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:00.986462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.9.39.188:33429)
0927 10:53:00.986587 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.986592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.988577 (+ 1985us) server_negotiation.cc:407] Connection header received
0927 10:53:00.988779 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.988783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.988834 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.988886 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.989534 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57520: BlockingRecv error: recv got EOF from 127.0.0.1:57520 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260927 10:53:00.991098 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:00.987240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.9.39.189:40321)
0927 10:53:00.987351 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.987356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.990147 (+ 2791us) server_negotiation.cc:407] Connection header received
0927 10:53:00.990333 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.990336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.990384 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.990435 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.990977 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51890: BlockingRecv error: recv got EOF from 127.0.0.1:51890 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 10:53:00.992432 31699 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:00.987899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.9.39.190:46369)
0927 10:53:00.988011 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.988017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.991550 (+ 3533us) server_negotiation.cc:407] Connection header received
0927 10:53:00.991723 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.991726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.991781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.991829 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.992299 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57752: BlockingRecv error: recv got EOF from 127.0.0.1:57752 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260927 10:53:00.996843 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:00.994831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.9.39.189:40321)
0927 10:53:00.994947 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:00.994951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:00.995869 (+ 918us) server_negotiation.cc:407] Connection header received
0927 10:53:00.996084 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:00.996087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:00.996166 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:00.996228 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:00.996741 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51892: BlockingRecv error: recv got EOF from 127.0.0.1:51892 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260927 10:53:01.001950 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.000511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51906 (local address 127.9.39.189:40321)
0927 10:53:01.000611 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.000615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.001096 (+ 481us) server_negotiation.cc:407] Connection header received
0927 10:53:01.001286 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.001289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.001337 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.001386 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.001850 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51906: BlockingRecv error: recv got EOF from 127.0.0.1:51906 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 10:53:01.007817 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.006191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.9.39.189:40321)
0927 10:53:01.006312 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.006316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.006883 (+ 567us) server_negotiation.cc:407] Connection header received
0927 10:53:01.007077 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.007080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.007131 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.007183 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.007706 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51918: BlockingRecv error: recv got EOF from 127.0.0.1:51918 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 10:53:01.015816 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.014121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51926 (local address 127.9.39.189:40321)
0927 10:53:01.014277 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.014281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.014850 (+ 569us) server_negotiation.cc:407] Connection header received
0927 10:53:01.015046 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.015049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.015103 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.015156 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.015702 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51926: BlockingRecv error: recv got EOF from 127.0.0.1:51926 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 10:53:01.028079 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.026309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.9.39.189:40321)
0927 10:53:01.026433 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.026438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.027068 (+ 630us) server_negotiation.cc:407] Connection header received
0927 10:53:01.027296 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.027300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.027356 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.027412 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.027961 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51936: BlockingRecv error: recv got EOF from 127.0.0.1:51936 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260927 10:53:01.049243 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.047350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.9.39.189:40321)
0927 10:53:01.047481 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.047488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.048103 (+ 615us) server_negotiation.cc:407] Connection header received
0927 10:53:01.048398 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.048402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.048464 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.048523 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.049110 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51944: BlockingRecv error: recv got EOF from 127.0.0.1:51944 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260927 10:53:01.085131 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.083835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51948 (local address 127.9.39.189:40321)
0927 10:53:01.083981 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.083986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.084229 (+ 243us) server_negotiation.cc:407] Connection header received
0927 10:53:01.084413 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.084417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.084465 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.084559 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.085009 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51948: BlockingRecv error: recv got EOF from 127.0.0.1:51948 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260927 10:53:01.152601 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.151419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.9.39.189:40321)
0927 10:53:01.151574 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.151581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.151699 (+ 118us) server_negotiation.cc:407] Connection header received
0927 10:53:01.151852 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.151855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.151904 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.152012 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.152464 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51956: BlockingRecv error: recv got EOF from 127.0.0.1:51956 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 10:53:01.284421 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.282998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.9.39.189:40321)
0927 10:53:01.283143 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.283146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.283341 (+ 195us) server_negotiation.cc:407] Connection header received
0927 10:53:01.283457 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.283459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.283511 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.283597 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.284178 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51972: BlockingRecv error: recv got EOF from 127.0.0.1:51972 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260927 10:53:01.543546 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:01.542585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.9.39.189:40321)
0927 10:53:01.542773 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:01.542777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:01.542793 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:53:01.542932 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:01.542935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:01.542981 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:01.543061 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:01.543448 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51978: BlockingRecv error: recv got EOF from 127.0.0.1:51978 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 10:53:01.872527 31881 catalog_manager.cc:1259] Loaded cluster ID: f39803867e8c430dadce43a599a633c6
I20260927 10:53:01.872605 31881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: loading cluster ID for follower catalog manager: success
I20260927 10:53:01.874332 31881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: acquiring CA information for follower catalog manager: success
I20260927 10:53:01.875912 31881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e735698e1ddf4a73b4fab37d343cdb92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 10:53:01.880781 31900 catalog_manager.cc:1259] Loaded cluster ID: f39803867e8c430dadce43a599a633c6
I20260927 10:53:01.880836 31900 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: loading cluster ID for follower catalog manager: success
I20260927 10:53:01.882453 31900 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: acquiring CA information for follower catalog manager: success
I20260927 10:53:01.883769 31900 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 79bc6a3100a341169ac8bba522cfbb04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 10:53:02.061817 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:02.057732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.9.39.189:40321)
0927 10:53:02.057974 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:02.057979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:02.058112 (+ 133us) server_negotiation.cc:407] Connection header received
0927 10:53:02.058282 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:02.058285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:02.058424 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:02.058561 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:02.058859 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.059396 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.060277 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.060513 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.060627 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:02.060963 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:02.061391 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:02.061533 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:02.061595 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 10:53:02.061652 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260927 10:53:02.069142 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:02.064980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57536 (local address 127.9.39.188:33429)
0927 10:53:02.065424 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:02.065427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:02.065441 (+ 14us) server_negotiation.cc:407] Connection header received
0927 10:53:02.065519 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:02.065521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:02.065720 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:02.065866 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:02.066097 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.066641 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.067663 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.067881 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.067961 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:02.068204 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:02.068704 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:02.068801 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:02.068887 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 10:53:02.068935 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":88,"threads_started":1}
I20260927 10:53:02.076527 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:02.072431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51994 (local address 127.9.39.189:40321)
0927 10:53:02.072575 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:02.072579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:02.072800 (+ 221us) server_negotiation.cc:407] Connection header received
0927 10:53:02.072947 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:02.072950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:02.073004 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:02.073074 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:02.073538 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.074058 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.075350 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.075614 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.075751 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:02.076030 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:02.076122 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:02.076192 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:02.076294 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 10:53:02.076354 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 10:53:02.077313 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:02.072249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57540 (local address 127.9.39.188:33429)
0927 10:53:02.072362 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:02.072367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:02.072847 (+ 480us) server_negotiation.cc:407] Connection header received
0927 10:53:02.073020 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:02.073023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:02.073078 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:02.073141 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:02.073597 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.074365 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.076149 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.076463 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.076553 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:02.076845 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:02.076910 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:02.076956 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:02.077140 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 10:53:02.077189 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:53:02.077648 31993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:02.072166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.9.39.190:46369)
0927 10:53:02.072429 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:02.072434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:02.072535 (+ 101us) server_negotiation.cc:407] Connection header received
0927 10:53:02.072749 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:02.072752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:02.072949 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:02.073079 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:02.073358 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.074304 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.075824 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:02.076148 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:02.076240 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:02.076624 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:02.076981 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:02.077071 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:02.077442 (+ 371us) server_negotiation.cc:299] Negotiation successful
0927 10:53:02.077506 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260927 10:53:02.080678 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31628
I20260927 10:53:02.087675 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31703
I20260927 10:53:02.093680 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 31778
2026-09-27T10:53:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3069 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 10:53:02 dist-test-slave-p9q6 krb5kdc[31997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 10:53:02 dist-test-slave-p9q6 krb5kdc[31997](info): set up 2 sockets
Sep 27 10:53:02 dist-test-slave-p9q6 krb5kdc[31997](info): commencing operation
krb5kdc: starting...
W20260927 10:53:03.127153 9374 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/test-user.keytab.
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T10:53:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T10:53:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.39.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
Entry for principal kudu/127.9.39.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab.
I20260927 10:53:03.255091 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40411
--webserver_interface=127.9.39.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40411,127.9.39.189:38315,127.9.39.188:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 10:53:03.367715 32010 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 10:53:03.368028 32010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:53:03.368053 32010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:53:03.368093 32010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:53:03.372084 32010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:53:03.372138 32010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:53:03.372156 32010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:53:03.372172 32010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:53:03.372187 32010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:53:03.377494 32010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40411,127.9.39.189:38315,127.9.39.188:35515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.190.keytab
--principal=kudu/127.9.39.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.190:40411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.39.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:53:03.378469 32010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:53:03.379434 32010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:53:03.388190 32016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:03.388206 32010 server_base.cc:1034] running on GCE node
W20260927 10:53:03.388329 32015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:03.388270 32018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:03.389032 32010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:53:03.390235 32010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:53:03.391426 32010 hybrid_clock.cc:648] HybridClock initialized: now 1790506383391408 us; error 31 us; skew 500 ppm
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:53:03.394089 32010 init.cc:377] Logged in from keytab as kudu/127.9.39.190@KRBTEST.COM (short username kudu)
I20260927 10:53:03.395025 32010 webserver.cc:460] Webserver started at http://127.9.39.190:46013/ using document root <none> and password file <none>
I20260927 10:53:03.395579 32010 fs_manager.cc:359] Metadata directory not provided
I20260927 10:53:03.395669 32010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:53:03.395916 32010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:53:03.397974 32010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data/instance:
uuid: "09e395b817664ac18031b8eaccfb72c7"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "5046e87a18f004c554271dd20809a63e"
server_key_iv: "f9b8faed4c9648f5f4bd4ed4c923374a"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.398541 32010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal/instance:
uuid: "09e395b817664ac18031b8eaccfb72c7"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "5046e87a18f004c554271dd20809a63e"
server_key_iv: "f9b8faed4c9648f5f4bd4ed4c923374a"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.402405 32010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260927 10:53:03.405141 32025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:53:03.406461 32010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 10:53:03.406639 32010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal
uuid: "09e395b817664ac18031b8eaccfb72c7"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "5046e87a18f004c554271dd20809a63e"
server_key_iv: "f9b8faed4c9648f5f4bd4ed4c923374a"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.406836 32010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:53:03.421205 32010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:53:03.424180 32010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:53:03.424393 32010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:53:03.431339 32010 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.190:40411
I20260927 10:53:03.431622 32077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.190:40411 every 8 connection(s)
I20260927 10:53:03.432441 32010 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/data/info.pb
I20260927 10:53:03.435328 32078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:53:03.439327 32078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:03.441996 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 32010
I20260927 10:53:03.442166 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-0/wal/instance
I20260927 10:53:03.442504 9374 external_mini_cluster.cc:1428] Setting key 7a6cc25032da2eef7e0d37f822238c14
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.39.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.189@KRBTEST.COM" created.
I20260927 10:53:03.457553 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.441775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.9.39.190:40411)
0927 10:53:03.442478 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.442497 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.442712 (+ 215us) server_negotiation.cc:407] Connection header received
0927 10:53:03.443802 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.443810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.444126 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.444312 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.445498 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.446373 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.448153 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.448548 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.448686 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.451799 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.451806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.451819 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.451843 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.453788 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.454465 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.454473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.454475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.454643 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.455038 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.455045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.455049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.455694 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.455856 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.456463 (+ 607us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.456693 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":544,"spinlock_wait_cycles":38272,"thread_start_us":174,"threads_started":1}
I20260927 10:53:03.457561 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.441775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40411 (local address 127.0.0.1:33188)
0927 10:53:03.442469 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.442522 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.443450 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.444374 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.444383 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.444909 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.445324 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.445348 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.446556 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.446563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.448004 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.448021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.448745 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.448751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.448914 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.449599 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.449625 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.451606 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.453916 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.453928 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.453942 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.454312 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.454796 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.454799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.454800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.454929 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.455898 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.455908 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.456107 (+ 199us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.456331 (+ 224us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.456777 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":63,"thread_start_us":148,"threads_started":1}
I20260927 10:53:03.460108 32078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:03.461011 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.460641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42740)
0927 10:53:03.460810 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.460910 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260927 10:53:03.461256 32028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
I20260927 10:53:03.462188 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.461898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42756)
0927 10:53:03.462043 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.462099 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260927 10:53:03.462406 32078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:38315: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab.
Entry for principal kudu/127.9.39.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab.
I20260927 10:53:03.474920 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:38315
--webserver_interface=127.9.39.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab
--principal=kudu/127.9.39.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40411,127.9.39.189:38315,127.9.39.188:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 10:53:03.511700 32078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } attempt: 1
I20260927 10:53:03.512737 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.512390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42766)
0927 10:53:03.512561 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.512636 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 10:53:03.513574 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.513316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42772)
0927 10:53:03.513442 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.513501 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 10:53:03.513770 32078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:38315: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
I20260927 10:53:03.582005 32078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } attempt: 2
I20260927 10:53:03.583006 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.582612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42788)
0927 10:53:03.582853 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.582918 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260927 10:53:03.583806 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.583563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42794)
0927 10:53:03.583695 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.583750 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260927 10:53:03.583940 32078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:38315: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
W20260927 10:53:03.595876 32085 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 10:53:03.596393 32085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:53:03.596447 32085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:53:03.596514 32085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:53:03.602403 32085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:53:03.602490 32085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:53:03.602511 32085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:53:03.602528 32085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:53:03.602545 32085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:53:03.608608 32085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40411,127.9.39.189:38315,127.9.39.188:35515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.189.keytab
--principal=kudu/127.9.39.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.189:38315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.39.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:53:03.609845 32085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:53:03.611060 32085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:53:03.620143 32093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:03.620134 32090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:03.620260 32085 server_base.cc:1034] running on GCE node
W20260927 10:53:03.620134 32091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:03.621095 32085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:53:03.622376 32085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:53:03.623569 32085 hybrid_clock.cc:648] HybridClock initialized: now 1790506383623511 us; error 74 us; skew 500 ppm
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:53:03.626873 32085 init.cc:377] Logged in from keytab as kudu/127.9.39.189@KRBTEST.COM (short username kudu)
I20260927 10:53:03.628010 32085 webserver.cc:460] Webserver started at http://127.9.39.189:36647/ using document root <none> and password file <none>
I20260927 10:53:03.628684 32085 fs_manager.cc:359] Metadata directory not provided
I20260927 10:53:03.628793 32085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:53:03.629020 32085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:53:03.631381 32085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data/instance:
uuid: "035406c2ff1943d69a4c6a15ef453198"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "7715e87d7a6ef12740326ec9e679a245"
server_key_iv: "d0f59d2ce3bb58f4ae205dace82e7dda"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.632018 32085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal/instance:
uuid: "035406c2ff1943d69a4c6a15ef453198"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "7715e87d7a6ef12740326ec9e679a245"
server_key_iv: "d0f59d2ce3bb58f4ae205dace82e7dda"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.636116 32085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 10:53:03.639209 32100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:53:03.640734 32085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 10:53:03.640931 32085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal
uuid: "035406c2ff1943d69a4c6a15ef453198"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "7715e87d7a6ef12740326ec9e679a245"
server_key_iv: "d0f59d2ce3bb58f4ae205dace82e7dda"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.641083 32085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:53:03.675127 32078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } attempt: 3
I20260927 10:53:03.676043 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.675734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42800)
0927 10:53:03.675899 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.675966 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260927 10:53:03.676848 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.676588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42812)
0927 10:53:03.676724 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.676779 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260927 10:53:03.676990 32078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.189:38315: Network error: Client connection negotiation failed: client connection to 127.9.39.189:38315: connect: Connection refused (error 111)
I20260927 10:53:03.680451 32085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:53:03.683561 32085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:53:03.683775 32085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:53:03.691200 32085 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.189:38315
I20260927 10:53:03.691486 32152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.189:38315 every 8 connection(s)
I20260927 10:53:03.692305 32085 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/data/info.pb
I20260927 10:53:03.692711 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 32085
I20260927 10:53:03.692876 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-1/wal/instance
I20260927 10:53:03.693202 9374 external_mini_cluster.cc:1428] Setting key 5d3fc2575044db0d6a1844e3cc53886f
I20260927 10:53:03.695479 32153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:53:03.699496 32153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.39.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.39.188@KRBTEST.COM" created.
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:53:03.715487 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.701655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.9.39.190:40411)
0927 10:53:03.701826 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.701831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.702739 (+ 908us) server_negotiation.cc:407] Connection header received
0927 10:53:03.703971 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.703977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.704048 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.704149 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.705362 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.705889 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.707791 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.708075 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.708190 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.711694 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.711698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.711700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.711728 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.713282 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.713814 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.713817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.713818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.713896 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.714243 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.714245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.714246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.714423 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.714493 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.715222 (+ 729us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.715353 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 10:53:03.716324 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.702074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40411 (local address 127.0.0.1:33202)
0927 10:53:03.702555 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.702600 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.703732 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.704313 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.704321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.704687 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.705193 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.705206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.706029 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.706035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.707606 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.707623 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.708270 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.708278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.708532 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.709301 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.709331 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.711420 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.713414 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.713420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.713433 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.713705 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.713979 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.713981 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.713982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.714119 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.714532 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.714541 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.714916 (+ 375us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.715134 (+ 218us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.715454 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":153,"threads_started":1}
I20260927 10:53:03.717864 32153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.39.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab.
Entry for principal kudu/127.9.39.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab.
I20260927 10:53:03.723358 9374 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasku3mICI/build/debug/bin/kudu
/tmp/dist-test-tasku3mICI/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:35515
--webserver_interface=127.9.39.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab
--principal=kudu/127.9.39.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.39.148:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.39.190:40411,127.9.39.189:38315,127.9.39.188:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:53:03.728170 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.718425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42824)
0927 10:53:03.718537 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.718555 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.718640 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.719258 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.719260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.719399 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.719491 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.719496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.720586 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.720588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.721230 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.721239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.721957 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.721960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.722042 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.722584 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.722604 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.724170 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.726535 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.726538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.726539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.726803 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.727114 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.727117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.727118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.727191 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.727784 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.727786 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.727865 (+ 79us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.727942 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.728046 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 10:53:03.728520 32158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.718507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.9.39.189:38315)
0927 10:53:03.718866 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.718888 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.718908 (+ 20us) server_negotiation.cc:407] Connection header received
0927 10:53:03.718942 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.718945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.719096 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.719299 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.719587 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.720465 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.721344 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.721838 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.721921 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.724327 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.724337 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.724351 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.724372 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.726418 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.726909 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.726918 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.726923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.727021 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.727288 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.727295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.727299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.727665 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.727835 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.728083 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.728263 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":156,"threads_started":1}
I20260927 10:53:03.730595 32153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:03.731432 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.731089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40896)
0927 10:53:03.731244 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.731343 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260927 10:53:03.731719 32101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
I20260927 10:53:03.732690 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.732410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40910)
0927 10:53:03.732549 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.732614 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260927 10:53:03.732858 32153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:35515: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
I20260927 10:53:03.782169 32153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } attempt: 1
I20260927 10:53:03.783124 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.782798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40916)
0927 10:53:03.782962 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.783034 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 10:53:03.783957 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.783697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40930)
0927 10:53:03.783819 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.783875 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 10:53:03.784149 32153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:35515: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
I20260927 10:53:03.820163 32078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } attempt: 4
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:53:03.830188 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.820783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42828)
0927 10:53:03.820974 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.820993 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.821096 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.821398 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.821400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.821573 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.821679 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.821685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.823153 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.823155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.824103 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.824113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.824738 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.824740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.824823 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.825372 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.825384 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.826838 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.828704 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.828707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.828708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.828909 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.829234 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.829238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.829244 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.829362 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.829785 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.829787 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.829874 (+ 87us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.829952 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.830062 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260927 10:53:03.830308 32158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.820879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.9.39.189:38315)
0927 10:53:03.821022 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.821027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.821050 (+ 23us) server_negotiation.cc:407] Connection header received
0927 10:53:03.821200 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.821203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.821261 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.821369 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.821806 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.823042 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.824242 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.824610 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.824732 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.826980 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.826985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.826989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.827018 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.828588 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.829011 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.829015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.829017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.829117 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.829457 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.829461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.829462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.829681 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.829792 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.830035 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.830159 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 10:53:03.831508 32078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } has no permanent_uuid. Determining permanent_uuid...
I20260927 10:53:03.832232 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.832002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40942)
0927 10:53:03.832121 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.832171 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 10:53:03.832872 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.832684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40946)
0927 10:53:03.832776 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.832822 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 10:53:03.833073 32078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:35515: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
W20260927 10:53:03.842154 32159 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 10:53:03.842461 32159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 10:53:03.842550 32159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 10:53:03.842619 32159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 10:53:03.847064 32159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 10:53:03.847133 32159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 10:53:03.847153 32159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 10:53:03.847172 32159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 10:53:03.847186 32159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 10:53:03.852389 32153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } attempt: 2
I20260927 10:53:03.852766 32159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.39.148:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.39.190:40411,127.9.39.189:38315,127.9.39.188:35515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/krb5kdc/kudu_127.9.39.188.keytab
--principal=kudu/127.9.39.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.39.188:35515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.39.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 10:05:23 UTC on ccycloud.quasar-frhndg.root.comops.site
build id 4675884
I20260927 10:53:03.853335 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.853008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40954)
0927 10:53:03.853175 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.853249 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 10:53:03.854144 32159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 10:53:03.854185 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.853948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40956)
0927 10:53:03.854062 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.854117 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 10:53:03.854346 32153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:35515: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
I20260927 10:53:03.855320 32159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 10:53:03.864056 32165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:03.864041 32166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 10:53:03.864044 32168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 10:53:03.864460 32159 server_base.cc:1034] running on GCE node
I20260927 10:53:03.865082 32159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 10:53:03.866250 32159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 10:53:03.867461 32159 hybrid_clock.cc:648] HybridClock initialized: now 1790506383867445 us; error 32 us; skew 500 ppm
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 10:53:03.870381 32159 init.cc:377] Logged in from keytab as kudu/127.9.39.188@KRBTEST.COM (short username kudu)
I20260927 10:53:03.871423 32159 webserver.cc:460] Webserver started at http://127.9.39.188:33289/ using document root <none> and password file <none>
I20260927 10:53:03.871995 32159 fs_manager.cc:359] Metadata directory not provided
I20260927 10:53:03.872061 32159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 10:53:03.872287 32159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 10:53:03.874380 32159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data/instance:
uuid: "eacbd39bb04d4e0898cb079e4e31e39c"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "5c64234fdb2c7fe10ca74234e96a15c1"
server_key_iv: "fbe32610b82f3492acd2abddaa8ec4d8"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.874989 32159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal/instance:
uuid: "eacbd39bb04d4e0898cb079e4e31e39c"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "5c64234fdb2c7fe10ca74234e96a15c1"
server_key_iv: "fbe32610b82f3492acd2abddaa8ec4d8"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.878894 32159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 10:53:03.881443 32175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 10:53:03.882845 32159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 10:53:03.882979 32159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data,/tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal
uuid: "eacbd39bb04d4e0898cb079e4e31e39c"
format_stamp: "Formatted at 2026-09-27 10:53:03 on dist-test-slave-p9q6"
server_key: "5c64234fdb2c7fe10ca74234e96a15c1"
server_key_iv: "fbe32610b82f3492acd2abddaa8ec4d8"
server_key_version: "encryptionkey@0"
I20260927 10:53:03.883105 32159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 10:53:03.892239 32078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } attempt: 1
I20260927 10:53:03.893077 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.892802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40960)
0927 10:53:03.892942 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.893002 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 10:53:03.893848 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.893576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:40970)
0927 10:53:03.893682 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.893767 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260927 10:53:03.893998 32078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.39.188:35515: Network error: Client connection negotiation failed: client connection to 127.9.39.188:35515: connect: Connection refused (error 111)
I20260927 10:53:03.896227 32159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 10:53:03.899353 32159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 10:53:03.899556 32159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 10:53:03.906944 32159 rpc_server.cc:307] RPC server started. Bound to: 127.9.39.188:35515
I20260927 10:53:03.907219 32227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.39.188:35515 every 8 connection(s)
I20260927 10:53:03.908027 32159 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/data/info.pb
I20260927 10:53:03.910902 9374 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasku3mICI/build/debug/bin/kudu as pid 32159
I20260927 10:53:03.911057 9374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasku3mICI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790506104903010-9374-0/minicluster-data/master-2/wal/instance
I20260927 10:53:03.911131 32228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 10:53:03.911381 9374 external_mini_cluster.cc:1428] Setting key 764e0965f10655cb268d681ec3403feb
I20260927 10:53:03.915964 32228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:53:03.926548 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.912836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.9.39.188:35515)
0927 10:53:03.913264 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.913281 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.913318 (+ 37us) server_negotiation.cc:407] Connection header received
0927 10:53:03.913991 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.914008 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.914354 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.914699 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.915542 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.917029 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.918009 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.918748 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.918901 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.921514 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.921519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.921529 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.921557 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.923413 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.923905 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.923911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.923921 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.924035 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.924344 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.924347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.924349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.924761 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.924981 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.925270 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.925623 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":122,"threads_started":1}
I20260927 10:53:03.927615 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.917929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.9.39.190:40411)
0927 10:53:03.918069 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.918074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.918413 (+ 339us) server_negotiation.cc:407] Connection header received
0927 10:53:03.918542 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.918545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.918588 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.918658 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.919424 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.919908 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.921147 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.921367 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.921482 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.923983 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.923986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.923989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.924017 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.925742 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.926237 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.926242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.926244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.926328 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.926611 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.926616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.926618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.926905 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.927018 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.927336 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.927462 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 10:53:03.927629 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.917796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.190:40411 (local address 127.0.0.1:33214)
0927 10:53:03.918272 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.918323 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.918440 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.918770 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.918779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.919206 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.919322 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.919327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.920035 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.920038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.921030 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.921041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.921515 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.921519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.921650 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.922259 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.922271 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.923822 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.925882 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.925893 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.925899 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.926129 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.926430 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.926433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.926435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.926518 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.927016 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.927023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.927137 (+ 114us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.927281 (+ 144us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.927463 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":155,"threads_started":1}
I20260927 10:53:03.928987 32228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.190@KRBTEST.COM
I20260927 10:53:03.937861 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.929548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.189:38315 (local address 127.0.0.1:42836)
0927 10:53:03.929656 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.929670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.929747 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.930013 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.930015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.930201 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.930300 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.930305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.931118 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.931124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.931638 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.931646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.932109 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.932113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.932217 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.932723 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.932741 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.934062 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.936273 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.936276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.936278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.936485 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.936787 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.936791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.936793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.936880 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.937301 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.937304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.937408 (+ 104us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.937519 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.937669 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 10:53:03.937861 32158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.929672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.9.39.189:38315)
0927 10:53:03.929807 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.929814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.929827 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:53:03.929867 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.929869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.929912 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.929998 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.930415 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.931010 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.931780 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.931997 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.932109 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.934229 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.934232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.934234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.934259 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.936159 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.936581 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.936586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.936591 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.936677 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.936984 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.936987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.936989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.937184 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.937308 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.937590 (+ 282us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.937704 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 10:53:03.938146 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.929275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.9.39.190:40411)
0927 10:53:03.929678 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.929682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.929695 (+ 13us) server_negotiation.cc:407] Connection header received
0927 10:53:03.929741 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.929744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.929785 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.929881 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.930537 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.931129 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.931918 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.932142 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.932254 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.934902 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.934906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.934909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.934940 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.936559 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.936957 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.936962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.936964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.937056 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.937335 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.937339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.937341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.937524 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.937785 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.937867 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.937978 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260927 10:53:03.939018 32228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.188@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.39.189@KRBTEST.COM
I20260927 10:53:03.945505 32153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } attempt: 3
I20260927 10:53:03.948698 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.939511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:41000)
0927 10:53:03.939808 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.939820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.939884 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.940099 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.940103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.940279 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.940390 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.940397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.941456 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.941459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.942280 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.942289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.942903 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.942906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.942991 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.943545 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.943561 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.945046 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.947024 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.947030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.947036 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.947293 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.947573 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.947575 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.947577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.947662 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.948122 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.948125 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.948252 (+ 127us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.948359 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.948483 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":1}
I20260927 10:53:03.949185 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.939699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.9.39.188:35515)
0927 10:53:03.939828 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.939833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.939849 (+ 16us) server_negotiation.cc:407] Connection header received
0927 10:53:03.939957 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.939959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.940000 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.940056 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.940497 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.941342 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.942402 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.942785 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.942890 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.945238 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.945245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.945251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.945278 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.946943 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.947392 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.947399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.947401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.947481 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.947761 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.947764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.947765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.948011 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.948121 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.948924 (+ 803us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.949059 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:53:03.950129 32158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.939407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.9.39.189:38315)
0927 10:53:03.939508 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.939532 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.939549 (+ 17us) server_negotiation.cc:407] Connection header received
0927 10:53:03.939717 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.939720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.939772 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.939853 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.940615 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.941348 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.942402 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.942795 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.942890 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.945889 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.945895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.945898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.945923 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.947860 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.948367 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.948371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.948373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.948459 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.949019 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.949023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.949024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.949223 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.949503 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.949873 (+ 370us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.949991 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 10:53:03.952672 32228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: Bootstrap starting.
I20260927 10:53:03.953655 9374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 10:53:03.955121 32228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: Neither blocks nor log segments found. Creating new log.
I20260927 10:53:03.954861 32052 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "09e395b817664ac18031b8eaccfb72c7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33230
I20260927 10:53:03.955852 32228 log.cc:824] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: Log is configured to *not* fsync() on all Append() calls
I20260927 10:53:03.957963 32228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: No bootstrap required, opened a new log
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.189@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
I20260927 10:53:03.961163 32078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } attempt: 2
I20260927 10:53:03.962281 32228 raft_consensus.cc:348] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.962550 32228 raft_consensus.cc:729] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eacbd39bb04d4e0898cb079e4e31e39c, State: Initialized, Role: FOLLOWER
I20260927 10:53:03.962735 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.946151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.9.39.188:35515)
0927 10:53:03.946632 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.946641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.946666 (+ 25us) server_negotiation.cc:407] Connection header received
0927 10:53:03.946739 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.946742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.947045 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.947143 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.947627 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.948949 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.949906 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.956561 (+ 6655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.956687 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.958991 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.958995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.958997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.959018 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.960549 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.961005 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.961007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.961009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.961093 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.961499 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.961503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.961506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.961880 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.962078 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.962341 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.962446 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":400,"thread_start_us":163,"threads_started":1}
I20260927 10:53:03.963001 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.945973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:41006)
0927 10:53:03.946490 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.946505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.946590 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.947176 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.947182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.947428 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.947533 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.947539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.949059 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.949063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.949748 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.949758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.956744 (+ 6986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.956747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.956891 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.957442 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.957455 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.958839 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.960650 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.960654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.960656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.960908 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.961249 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.961259 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.961261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.961370 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.962050 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.962053 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.962185 (+ 132us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.962295 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.962447 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428}
I20260927 10:53:03.962986 32228 consensus_queue.cc:260] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.963476 32235 sys_catalog.cc:455] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:03.963611 32235 sys_catalog.cc:458] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:03.964366 32228 sys_catalog.cc:564] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:53:03.966370 32153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: Bootstrap starting.
Sep 27 10:53:03 dist-test-slave-p9q6 krb5kdc[31997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790506383, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.39.190@KRBTEST.COM for kudu/127.9.39.188@KRBTEST.COM
W20260927 10:53:03.969655 32246 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:53:03.969727 32246 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:53:03.971473 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.961587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.39.188:35515 (local address 127.0.0.1:41014)
0927 10:53:03.961771 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0927 10:53:03.961789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 10:53:03.961898 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 10:53:03.962187 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 10:53:03.962190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 10:53:03.962389 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0927 10:53:03.962498 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.962507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.963896 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.963900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.964662 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 10:53:03.964674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.965344 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.965348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 10:53:03.965460 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.965984 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 10:53:03.966001 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 10:53:03.967648 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 10:53:03.969700 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.969704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.969706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.969965 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.970306 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 10:53:03.970309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 10:53:03.970311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 10:53:03.970410 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 10:53:03.970908 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 10:53:03.970912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 10:53:03.971024 (+ 112us) client_negotiation.cc:769] Sending connection context
0927 10:53:03.971142 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 10:53:03.971311 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260927 10:53:03.971500 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:03.961604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.9.39.188:35515)
0927 10:53:03.961726 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:03.961731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:03.961847 (+ 116us) server_negotiation.cc:407] Connection header received
0927 10:53:03.961993 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:03.961996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:03.962040 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:03.962149 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:03.962657 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.963775 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.964803 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:03.965164 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:03.965729 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:03.967803 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:03.967806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:03.967809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:03.967831 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:03.969571 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.970090 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.970093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.970095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.970194 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:03.970502 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:03.970505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:03.970507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:03.970805 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:03.970902 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:03.971234 (+ 332us) server_negotiation.cc:299] Negotiation successful
0927 10:53:03.971350 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 10:53:03.968989 32159 master_runner.cc:186] Error getting master registration for 127.9.39.190:40411: OK, instance_id { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" instance_seqno: 1790506383429383 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 10:53:03.972497 32159 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:53:03.974680 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7: Bootstrap starting.
I20260927 10:53:03.974848 32153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: Neither blocks nor log segments found. Creating new log.
I20260927 10:53:03.975746 32153 log.cc:824] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: Log is configured to *not* fsync() on all Append() calls
I20260927 10:53:03.977133 32078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7: Neither blocks nor log segments found. Creating new log.
I20260927 10:53:03.977916 32078 log.cc:824] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7: Log is configured to *not* fsync() on all Append() calls
I20260927 10:53:03.978380 32153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: No bootstrap required, opened a new log
I20260927 10:53:03.982538 32153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.982820 32153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035406c2ff1943d69a4c6a15ef453198, State: Initialized, Role: FOLLOWER
I20260927 10:53:03.983166 32153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.983673 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:03.983821 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:03.984683 32153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:53:03.985724 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7: No bootstrap required, opened a new log
W20260927 10:53:03.990435 32261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 10:53:03.990579 32261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 10:53:03.990620 32078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.990991 32078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e395b817664ac18031b8eaccfb72c7, State: Initialized, Role: FOLLOWER
I20260927 10:53:03.991506 32078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.988581 32085 master_runner.cc:186] Error getting master registration for 127.9.39.190:40411: OK, instance_id { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" instance_seqno: 1790506383429383 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 10:53:03.992055 32085 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:53:03.992472 32263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:03.992602 32263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:03.993258 32078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 10:53:03.993333 32052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 10:53:03.993435 32052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:53:03.994927 32052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:03.995982 32052 leader_election.cc:288] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [CANDIDATE]: Term 1 election: Requested vote from peers 035406c2ff1943d69a4c6a15ef453198 (127.9.39.189:38315), eacbd39bb04d4e0898cb079e4e31e39c (127.9.39.188:35515)
I20260927 10:53:03.996660 32202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09e395b817664ac18031b8eaccfb72c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eacbd39bb04d4e0898cb079e4e31e39c"
I20260927 10:53:03.996944 32202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:53:03.997006 32127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09e395b817664ac18031b8eaccfb72c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "035406c2ff1943d69a4c6a15ef453198"
I20260927 10:53:03.997221 32127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [term 0 FOLLOWER]: Advancing to term 1
I20260927 10:53:03.998376 32202 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e395b817664ac18031b8eaccfb72c7 in term 1.
I20260927 10:53:03.998485 32127 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e395b817664ac18031b8eaccfb72c7 in term 1.
I20260927 10:53:03.999354 32026 leader_election.cc:302] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e395b817664ac18031b8eaccfb72c7, eacbd39bb04d4e0898cb079e4e31e39c; no voters:
I20260927 10:53:03.999698 32263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 10:53:03.999907 32263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [term 1 LEADER]: Becoming Leader. State: Replica: 09e395b817664ac18031b8eaccfb72c7, State: Running, Role: LEADER
I20260927 10:53:04.000290 32263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } }
I20260927 10:53:04.002179 32263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09e395b817664ac18031b8eaccfb72c7. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.002286 32263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:53:04.002725 32281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 10:53:04.006597 32010 master_runner.cc:263] No leader master found from master 09e395b817664ac18031b8eaccfb72c7
I20260927 10:53:04.006686 32010 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 10:53:04.013375 32281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 10:53:04.018636 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:04.004562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.9.39.190:40411)
0927 10:53:04.004687 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.004692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.006321 (+ 1629us) server_negotiation.cc:407] Connection header received
0927 10:53:04.006519 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.006524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.006575 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.006628 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.007741 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:04.008451 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:04.009587 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:04.009896 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:04.009997 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:04.014537 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:04.014542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:04.014545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:04.014576 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:04.016474 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:04.017240 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:04.017244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:04.017246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:04.017342 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:04.017631 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:04.017635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:04.017636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:04.017865 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:04.017971 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:04.018313 (+ 342us) server_negotiation.cc:299] Negotiation successful
0927 10:53:04.018456 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 10:53:04.025344 32158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:04.005356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.9.39.189:38315)
0927 10:53:04.005479 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.005485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.009991 (+ 4506us) server_negotiation.cc:407] Connection header received
0927 10:53:04.010807 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.010810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.010880 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.010945 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.011891 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:04.012696 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:04.013752 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:04.014056 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:04.014138 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:04.021308 (+ 7170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:04.021313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:04.021316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:04.021350 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:04.023310 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:04.023872 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:04.023896 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:04.023898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:04.023994 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:04.024309 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:04.024312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:04.024314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:04.024550 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:04.024653 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:04.025030 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 10:53:04.025164 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 10:53:04.031129 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:04.005715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.9.39.188:35515)
0927 10:53:04.005820 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.005824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.010244 (+ 4420us) server_negotiation.cc:407] Connection header received
0927 10:53:04.011107 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.011111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.011187 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.011262 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.016115 (+ 4853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:04.016888 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:04.018034 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:04.018403 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:04.018484 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:04.027338 (+ 8854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 10:53:04.027343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 10:53:04.027347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 10:53:04.027380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 10:53:04.029135 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:04.029633 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:04.029636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:04.029639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:04.029744 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 10:53:04.030048 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 10:53:04.030052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 10:53:04.030054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 10:53:04.030282 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 10:53:04.030393 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:04.030817 (+ 424us) server_negotiation.cc:299] Negotiation successful
0927 10:53:04.030958 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 10:53:04.031761 32202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [term 1 FOLLOWER]: Refusing update from remote peer 09e395b817664ac18031b8eaccfb72c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:53:04.032706 32275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:53:04.034062 32127 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [term 1 FOLLOWER]: Refusing update from remote peer 09e395b817664ac18031b8eaccfb72c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 10:53:04.036965 32235 sys_catalog.cc:455] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09e395b817664ac18031b8eaccfb72c7. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.037079 32235 sys_catalog.cc:458] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:04.037456 32275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 10:53:04.041704 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09e395b817664ac18031b8eaccfb72c7. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.041852 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:04.048036 32275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.048195 32275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:53:04.048877 32263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.049015 32263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7 [sys.catalog]: This master's current role is: LEADER
I20260927 10:53:04.050498 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.050624 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:04.051826 32287 mvcc.cc:205] Tried to move back new op lower bound from 7333914148984524800 to 7333914148868915200. Current Snapshot: MvccSnapshot[applied={T|T < 7333914148984524800}]
I20260927 10:53:04.052003 32235 sys_catalog.cc:455] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09e395b817664ac18031b8eaccfb72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e395b817664ac18031b8eaccfb72c7" member_type: VOTER last_known_addr { host: "127.9.39.190" port: 40411 } } peers { permanent_uuid: "035406c2ff1943d69a4c6a15ef453198" member_type: VOTER last_known_addr { host: "127.9.39.189" port: 38315 } } peers { permanent_uuid: "eacbd39bb04d4e0898cb079e4e31e39c" member_type: VOTER last_known_addr { host: "127.9.39.188" port: 35515 } } }
I20260927 10:53:04.052115 32235 sys_catalog.cc:458] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 10:53:04.052685 32281 catalog_manager.cc:1347] Generated new cluster ID: e109d541319542d9acd9f684736b52ec
I20260927 10:53:04.052755 32281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 10:53:04.092947 32281 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 10:53:04.094344 32281 catalog_manager.cc:1504] Loading token signing keys...
I20260927 10:53:04.108467 32281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09e395b817664ac18031b8eaccfb72c7: Generated new TSK 0
I20260927 10:53:04.109467 32281 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 10:53:04.148542 32081 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.145289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.9.39.190:40411)
0927 10:53:04.145425 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.145430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.147396 (+ 1966us) server_negotiation.cc:407] Connection header received
0927 10:53:04.147598 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.147601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.147653 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.147707 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.148396 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33248: BlockingRecv error: recv got EOF from 127.0.0.1:33248 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260927 10:53:04.149978 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.146159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.9.39.189:38315)
0927 10:53:04.146268 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.146273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.149052 (+ 2779us) server_negotiation.cc:407] Connection header received
0927 10:53:04.149241 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.149244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.149294 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.149345 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.149846 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42860: BlockingRecv error: recv got EOF from 127.0.0.1:42860 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260927 10:53:04.151341 32231 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.146800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.9.39.188:35515)
0927 10:53:04.146933 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.146937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.150444 (+ 3507us) server_negotiation.cc:407] Connection header received
0927 10:53:04.150608 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.150612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.150662 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.150748 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.151234 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41024: BlockingRecv error: recv got EOF from 127.0.0.1:41024 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260927 10:53:04.155802 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.153512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42870 (local address 127.9.39.189:38315)
0927 10:53:04.153615 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.153619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.154762 (+ 1143us) server_negotiation.cc:407] Connection header received
0927 10:53:04.154999 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.155002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.155076 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.155127 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.155695 (+ 568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42870: BlockingRecv error: recv got EOF from 127.0.0.1:42870 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 10:53:04.161020 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.159319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.9.39.189:38315)
0927 10:53:04.159449 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.159453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.160031 (+ 578us) server_negotiation.cc:407] Connection header received
0927 10:53:04.160231 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.160235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.160289 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.160342 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.160902 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42886: BlockingRecv error: recv got EOF from 127.0.0.1:42886 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 10:53:04.167829 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.166050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.9.39.189:38315)
0927 10:53:04.166165 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.166169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.166867 (+ 698us) server_negotiation.cc:407] Connection header received
0927 10:53:04.167071 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.167074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.167127 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.167179 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.167712 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42894: BlockingRecv error: recv got EOF from 127.0.0.1:42894 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 10:53:04.175772 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.174112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.9.39.189:38315)
0927 10:53:04.174250 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.174255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.174802 (+ 547us) server_negotiation.cc:407] Connection header received
0927 10:53:04.175017 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.175020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.175072 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.175123 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.175658 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42908: BlockingRecv error: recv got EOF from 127.0.0.1:42908 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260927 10:53:04.188444 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.186589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.9.39.189:38315)
0927 10:53:04.186731 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.186736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.187377 (+ 641us) server_negotiation.cc:407] Connection header received
0927 10:53:04.187617 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.187622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.187691 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.187795 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.188316 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42924: BlockingRecv error: recv got EOF from 127.0.0.1:42924 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260927 10:53:04.207938 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.206761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.9.39.189:38315)
0927 10:53:04.206911 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.206916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.207103 (+ 187us) server_negotiation.cc:407] Connection header received
0927 10:53:04.207217 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.207221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.207290 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.207398 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.207820 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42940: BlockingRecv error: recv got EOF from 127.0.0.1:42940 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260927 10:53:04.243552 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.242307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.9.39.189:38315)
0927 10:53:04.242461 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.242467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.242599 (+ 132us) server_negotiation.cc:407] Connection header received
0927 10:53:04.242755 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.242759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.242817 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.242925 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.243427 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42954: BlockingRecv error: recv got EOF from 127.0.0.1:42954 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 10:53:04.311266 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.309774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.9.39.189:38315)
0927 10:53:04.309976 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.309983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.310212 (+ 229us) server_negotiation.cc:407] Connection header received
0927 10:53:04.310296 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.310299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.310350 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.310429 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.311103 (+ 674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42970: BlockingRecv error: recv got EOF from 127.0.0.1:42970 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260927 10:53:04.442888 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.441544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42974 (local address 127.9.39.189:38315)
0927 10:53:04.441715 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.441721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.441919 (+ 198us) server_negotiation.cc:407] Connection header received
0927 10:53:04.442110 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.442113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.442162 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.442259 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.442773 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42974: BlockingRecv error: recv got EOF from 127.0.0.1:42974 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260927 10:53:04.702428 32158 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 10:53:04.701310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42984 (local address 127.9.39.189:38315)
0927 10:53:04.701460 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:04.701465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:04.701554 (+ 89us) server_negotiation.cc:407] Connection header received
0927 10:53:04.701701 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:04.701704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:04.701759 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:04.701844 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 10:53:04.702319 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42984: BlockingRecv error: recv got EOF from 127.0.0.1:42984 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 10:53:04.971081 32246 catalog_manager.cc:1259] Loaded cluster ID: e109d541319542d9acd9f684736b52ec
I20260927 10:53:04.971170 32246 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: loading cluster ID for follower catalog manager: success
I20260927 10:53:04.972932 32246 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: acquiring CA information for follower catalog manager: success
I20260927 10:53:04.974200 32246 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eacbd39bb04d4e0898cb079e4e31e39c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 10:53:04.991904 32261 catalog_manager.cc:1259] Loaded cluster ID: e109d541319542d9acd9f684736b52ec
I20260927 10:53:04.991989 32261 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: loading cluster ID for follower catalog manager: success
I20260927 10:53:04.993654 32261 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: acquiring CA information for follower catalog manager: success
I20260927 10:53:04.995043 32261 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 035406c2ff1943d69a4c6a15ef453198: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 10:53:05.220623 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:05.216782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.9.39.189:38315)
0927 10:53:05.217027 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:05.217031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:05.217127 (+ 96us) server_negotiation.cc:407] Connection header received
0927 10:53:05.217247 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:05.217249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:05.217374 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:05.217483 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:05.217768 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.218285 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.219249 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.219483 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.219565 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:05.219848 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:05.220281 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:05.220373 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:05.220454 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 10:53:05.220503 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260927 10:53:05.227865 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:05.223836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.9.39.188:35515)
0927 10:53:05.224130 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:05.224137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:05.224152 (+ 15us) server_negotiation.cc:407] Connection header received
0927 10:53:05.224419 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:05.224422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:05.224547 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:05.224686 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:05.224910 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.225439 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.226297 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.226530 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.226646 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:05.226984 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:05.227431 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:05.227567 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:05.227629 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 10:53:05.227698 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":83,"threads_started":1}
I20260927 10:53:05.235311 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:05.231261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.9.39.188:35515)
0927 10:53:05.231373 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:05.231377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:05.231682 (+ 305us) server_negotiation.cc:407] Connection header received
0927 10:53:05.231863 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:05.231866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:05.231913 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:05.231959 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:05.232327 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.233158 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.234206 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.234448 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.234561 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:05.234840 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:05.234918 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:05.235000 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:05.235086 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 10:53:05.235137 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 10:53:05.236239 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:05.231829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.9.39.189:38315)
0927 10:53:05.231997 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:05.232002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:05.232087 (+ 85us) server_negotiation.cc:407] Connection header received
0927 10:53:05.232234 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:05.232237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:05.232291 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:05.232368 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:05.232631 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.233142 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.234642 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.235023 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.235103 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:05.235759 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:05.235836 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:05.235877 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:05.236036 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 10:53:05.236103 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260927 10:53:05.236398 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 10:53:05.231262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.9.39.190:40411)
0927 10:53:05.231482 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 10:53:05.231486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 10:53:05.231588 (+ 102us) server_negotiation.cc:407] Connection header received
0927 10:53:05.231817 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 10:53:05.231822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 10:53:05.231993 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 10:53:05.232103 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 10:53:05.232465 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.233271 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.234642 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 10:53:05.235026 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 10:53:05.235114 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 10:53:05.235675 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 10:53:05.236044 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 10:53:05.236125 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 10:53:05.236197 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 10:53:05.236257 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260927 10:53:05.239450 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 32010
I20260927 10:53:05.245886 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 32085
I20260927 10:53:05.251902 9374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasku3mICI/build/debug/bin/kudu with pid 32159
2026-09-27T10:53:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3157 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6227 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (280343 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST